Bug#765854: eCryptfs in Buster / Bullseye (bug #765854, #936465)

[Daniel Lange]

Hi folks,

we have the issue that eCryptfs has not made it into Buster and has 
fallen out of testing due to bug #765854.


To me it seems the most easy solution is from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=765854#107
as non-interactive logins don't have any passphrase to unlock an 
encrypted home dir anyways.


Additionally we have #936465 now (Python2 dependency).
This is tracked upstream at
https://bugs.launchpad.net/ecryptfs/+bug/1871236

So questions:

@Martin, Dustin:
Is there still upstream support for eCryptfs?
I.e. will you resolve the LP bug linked above?
libecryptfs.py is just a SWIG generated wrapper.
So this probably trivial. But somebody that actually uses the 
python-bindings would be needed to test.


Any comments on the best solution from the ones offered in Debian bug 
#765854?


@Laszlo, Julian:
Do we want to get eCryptfs back into Bullseye so Stretch users can 
upgrade there (or may be document a work-around with testing packages, 
or we do a stable update for these folks)?

I'd really like to offer a solution to users of eCryptfs in Debian.

Some data from popcon:

Package Users
--- -
ecryptfs-utils   1278
libecryptfs1 1122
python-ecryptfs16

Kind regards,
Daniel

Bug#908678: Split file repo v2

[Daniel Lange]

as requested in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=908678#139
we have created a data/CVE/.list repo ("v2") during MiniDebConf HH

It is mirrored at Salsa:
https://salsa.debian.org/dlange/debian_security_security-tracker_split_files_v2

Bug#908678: Update on the security-tracker git discussion

[Daniel Lange]

Am 06.06.19 um 07:31 schrieb Salvatore Bonaccorso:

Could you again point me to your splitted up variant mirror?


https://git.faster-it.de/debian_security_security-tracker_split_files/

Bug#908678: Update on the security-tracker git discussion

[Daniel Lange]

Zobel brought up the security-tracker git discussion in the 
#debian-security irc channel again and I'd like to record a few of the 
items touched there for others that were not present:


DLange has a running mirror of the git repo with split files since three 
months. This is based on anarcat's scripts published previously in this 
bug. The rewriting mirror repo works flawlessly. All history is retained 
sans gpg commit signatures.


Corsac noted that "redoing the tooling is a pain" and anarcat and DLange 
iterated we are willing to help fix the tools. But we need a commitment 
from the security-team that the migration to a split file repo is 
wanted. And we need a prioritized list of tools that need to be 
split-files enabled.


The discussion iterated that "moving elsewhere" doesn't really fix the 
underlying git-usage issue. So while this would take load off salsa, it 
will not improve clone times and hamper collaboration with Debian people 
outside the security team.


Still - to gain some data - DLange tried to push the security-tracker 
repo to github. This bails out as the history contains a file > 100MB 
(hard limit for Github):


remote: error: GH001: Large files detected. You may want to try Git 
Large File Storage - https://git-lfs.github.com.

[..]
remote: error: File data/CVE/allitems.html is 111.44 MB; this exceeds 
GitHub's file size limit of 100.00 MB


So we would have to re-write history for pushing to GitHub. Commits from 
2017-12-29 that introduce "data/CVE/allitems.html" and drop it again 
would need to be modified. Technically all commits after these have to 
be re-written as well. I have not tested whether Github supports 
refs/replace substitutes which would be a work-around.


As noticeable on Salsa and per 
https://gitlab.com/gitlab-com/support-forum/issues/230 Gitlab does not 
enforce per-file size limits.
But the pain of hosting and using this repo is not really different for 
any Gitlab instance.


So that means self-hosting of a non-split-file repo would probably have 
to be on a security DSA machine or similar.


Again, as said above, discussion participants outside the security team 
would prefer a commitment to split the offending data/CVE/list file into 
annual chunks, enable the tooling and stay on salsa.

Bug#765854: Plans for Netatalk for Buster?

[Daniel Lange]

> Hi Jonas,
> 
> I was wondering what your plans are for Netatalk in Buster?
> 1) stay on 2.2.x
> 2) adopt the newer packages for 3.1.x (still maintained at
>https://github.com/adiknoth/netatalk-debian as per bug #690227)
> 3) RM netatalk

Please re-post these excellent questions to 690...@bugs.debian.org!


 - Jonas

Bug#765854: Related systemd upstream issue

[Daniel Lange]

https://github.com/systemd/systemd/issues/8598
(Title: systemd-user doesn't properly close its PAM session)

Bug#908678: Testing the filter-branch scripts

[Daniel Lange]

Am 13.11.18 um 23:09 schrieb Moritz Muehlenhoff:
> The current data structure works very well for us and splitting the files
> has many downsides.

Could you detail what those many downsides are besides the scripts that
need to be amended?

Bug#908678: Testing the filter-branch scripts

[Daniel Lange]

> The Python job finished successfully here after 10 hours.
6h40 mins here as I ported your improved logic to the python2 version :).

# git filter-branch --tree-filter '/usr/bin/python2 /split-by-year.pyc' HEAD
Rewrite 1169d256b27eb7244273671582cc08ba88002819 (68356/68357) (24226 seconds 
passed, remaining 0 predicted)
Ref 'refs/heads/master' was rewritten

The tree-filter blows up the .git/objects store to 13G though.
But nothing a git gc can't fix.

> 
> I did some tests on the new git repository. Cloning the repository from
> scratch takes around 2 minutes (the original repo: 21 minutes).
Confirmed.

> So that's about it. I have not done a thorough job at checking the
> actual *integrity* of the results. It's difficult, considering CVE
> identifiers are not sequential in the data/CVE/list file, so a naive
> diff like this will fail:
> 
> $ diff -u <(cat 
> ../security-tracker-full-test-filtered-bis/data/CVE/list.{2019,2018,2017,2016,2015,2014,2013,2012,2011,2010,2009,2008,2007,2006,2005,2004,2003,2002,2001,2000,1999}
>  ) data/CVE/list | diffstat
>  list |106562 
> +--
>  1 file changed, 53281 insertions(+), 53281 deletions(-)
> 
> But at least the numbers add up: it looks like no line is lost. And
> indeed, it looks like all CVEs add up:
> 
> $ diff -u <(cat 
> ../security-tracker-full-test-filtered-bis/data/CVE/list.{2019,2018,2017,2016,2015,2014,2013,2012,2011,2010,2009,2008,2007,2006,2005,2004,2003,2002,2001,2000,1999}
>  | grep ^CVE | sort -n ) <( grep ^CVE data/CVE/list | sort -n  ) | diffstat
>  0 files changed
> 
> A cursory look at the diff seems to indicate it is clean, however.

I uploaded "my" version to https://people.debian.org/~dlange/
so people can poke the log and diffs and see whether there are any
issues left.

> I looked at splitting that file per CVE. That did not scale and just
> created new problems. But splitting by *year* seems like a very
> efficient switch, and I think it would be worth pursuing that idea
> forward.

The tools in bin/ would need a brush through. I.e. throw away the
unused ones and amend the ones that are used on data/CVE/* to learn
about the split files.

Bug#908678: Testing the filter-branch scripts

[Daniel Lange]

Antoine,

thank you very much for your filter-branch scripts.

I tested each:

1) the golang version:
It completes after 3h36min:

# git filter-branch --tree-filter '/split-by-year' HEAD
Rewrite a09118bf0a33f3721c0b8f6880c4cbb1e407a39d (68282/68286) (12994 seconds 
passed, remaining 0 predicted)
Ref 'refs/heads/master' was rewritten

But it doesn't Close() the os.OpenFile handles so ...
all data/CVE/list. files are 0 bytes long. Sic!

I can reproduce that just running the golang executable
against a current checkout of data/CVE/list.

# go version
go version go1.10.3 linux/amd64
(Stretch backport golang-go 2:1.10~5~bpo9+1)

2.1) the Python version
You claim #!/usr/bin/python3 in the shebang, so I tried that first:

# git filter-branch --tree-filter '/usr/bin/python3 
/__pycache__/split-by-year.cpython-35.pyc' HEAD
Rewrite 990d3c4bbb49308fb3de1e0e91b9ba5600386f8a (1220/68293) (41 seconds 
passed, remaining 2254 predicted)
  Traceback (most recent call last):
  File "split-by-year.py", line 13, in 
  File "/usr/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 5463: 
invalid start byte
tree filter failed: /usr/bin/python3 /__pycache__/split-by-year.cpython-35.pyc

The offending commit is:
* 990d3c4bbb - Rename sarge-checks data to something not specific to sarge, 
since we're working on etch now.
  Sorry for the probable annoyance, but it had to be done. (13 years ago) [Joey 
Hess]

There will be many more like this, so for Python3
this needs needs to be made unicode-agnostic.

Notice I compiled the .py to .pyc which makes it
much faster and thus well usable.

2.2) Python, when a string was a string .. Python2
Your code is actually Python2, so why not give that a try:

# git filter-branch --tree-filter '/usr/bin/python2 /split-by-year.pyc' HEAD
Rewrite b59da20b82011ffcfa6c4a453de9df58ee036b2c (2516/68293) (113 seconds 
passed, remaining 2954 predicted)
  Traceback (most recent call last):
  File "split-by-year.py", line 18, in 
yearly = 'data/CVE/list.{:d}'.format(year)
NameError: name 'year' is not defined
tree filter failed: /usr/bin/python2 /split-by-year.pyc

The offending commit is:
* b59da20b82 - claim (13 years ago) [Moritz Muehlenhoff]
| diff --git a/data/CVE/list b/data/CVE/list
| index 7b5d1d21d6..cdf0b74dd0 100644
| --- a/data/CVE/list
| +++ b/data/CVE/list
| @@ -1,3 +1,4 @@
| +begin claimed by jmm
|  CVE-2005-3276 (The sys_get_thread_area function in process.c in Linux 2.6 
before ...)
|   TODO: check
|  CVE-2005-3275 (The NAT code (1) ip_nat_proto_tcp.c and (2) 
ip_nat_proto_udp.c in ...)
| @@ -34,6 +35,7 @@ CVE-2005-3260 (Multiple cross-site scripting (XSS) 
vulnerabilities in ...)
|   TODO: check
|  CVE-2005-3259 (Multiple SQL injection vulnerabilities in 
versatileBulletinBoard (vBB) ...)
|   TODO: check
| +end claimed by jmm
|  CVE-2005- [Insecure caching of user id in mantis]
|   - mantis  (bug #330682; unknown)
|  CVE-2005- [Filter information disclosure in mantis]

As you see the line "+begin claimed by jmm" breaks the too simplistic parser 
logic.
Unfortunately dry-running against a current version of data/CVE/list such 
errors do not show up.
The "violations" of the file format are transient and buried in history.

Best,
Daniel

Bug#908678: Some more thoughts and some tests on the security-tracker git repo

[Daniel Lange]

The main issue is that we need to get clone and diff+render operations
back into normal time frames. The salsa workers (e.g. to render a
diff) time out after 60s. Similar time constraints are put onto other
rendering frond-ends. Actually you can easily get Apache to segfault
if you do not time-constrain cgi/fcgi type processes.
But that's out of scope here.

Back on topic:

Just splitting the file will not do. We need to (unfortunately)
somehow "get rid" of the history (delta-resolution) walks in git:

# test setup limits: Network bw: 200 MBit, client system: 4 core

$ time git clone https://.../debian_security_security-tracker
Klone nach 'debian_security_security-tracker' ...
remote: Counting objects: 334274, done.
remote: Compressing objects: 100% (67288/67288), done.
remote: Total 334274 (delta 211939), reused 329399 (delta 208905)
Empfange Objekte: 100% (334274/334274), 165.46 MiB | 21.93 MiB/s, 
Fertig.
Löse Unterschiede auf: 100% (211939/211939), Fertig.

real14m13,159s
user27m23,980s
sys 0m17,068s

# Run the tool already available to split the main CVE/list
# file into annual files. Thanks Raphael Geissert!
$ bin/split-by-year

# remove the old big CVE/list file
$ git rm data/CVE/list

# get the new files into git
$ git add data/CVE/list.*
$ git commit --all
[master a06d3446ca] Remove list and commit bin/split-by-year results
 21 files changed, 342414 insertions(+), 342414 deletions(-)
 delete mode 100644 data/CVE/list
 create mode 100644 data/CVE/list.1999
 create mode 100644 data/CVE/list.2000
 create mode 100644 data/CVE/list.2001
 create mode 100644 data/CVE/list.2002
 create mode 100644 data/CVE/list.2003
 create mode 100644 data/CVE/list.2004
 create mode 100644 data/CVE/list.2005
 create mode 100644 data/CVE/list.2006
 create mode 100644 data/CVE/list.2007
 create mode 100644 data/CVE/list.2008
 create mode 100644 data/CVE/list.2009
 create mode 100644 data/CVE/list.2010
 create mode 100644 data/CVE/list.2011
 create mode 100644 data/CVE/list.2012
 create mode 100644 data/CVE/list.2013
 create mode 100644 data/CVE/list.2014
 create mode 100644 data/CVE/list.2015
 create mode 100644 data/CVE/list.2016
 create mode 100644 data/CVE/list.2017
 create mode 100644 data/CVE/list.2018

# this one is fast:
$ git push

# create a new clone
$ time git clone 
https://.../debian_security_security-tracker_split_files test-clone
Klone nach 'test-clone' ...
remote: Counting objects: 334298, done.
remote: Compressing objects: 100% (67312/67312), done.
remote: Total 334298 (delta 211943), reused 329399 (delta 208905)
Empfange Objekte: 100% (334298/334298), 168.91 MiB | 21.28 MiB/s, 
Fertig.
Löse Unterschiede auf: 100% (211943/211943), Fertig.

real14m35,444s
user27m45,500s
sys 0m21,100s

--> so splitting alone doesn't help. Git is not clever enough to not run
through the deltas of not to be checked-out files.

git 2.18's git2 wire protocol could be used with server-side filtering
but that's an awful hack. Telling people to

git clone --depth 1 #(shallow)

like Guido advises is easier and more reliable for the clone use-case.
For the original repo that will take ~1.5s, for a split-by-year repo ~0.2s.

There are tools to split git files and keep the history
e.g. https://github.com/potherca-bash/git-split-file
but we'd need (to create) one that also zaps the old deltas.
So really "rewrite history" as the git folks tend to call this.
git filter-branch can do this. But it would get somewhat complex and murky
with commits that span CVE/list-year and list-year+1 which are at least 21 for
2018+2017, 19 for 2017+2016 and ~10 for previous year combos.
So I wouldn't put too much effort into that path.

In any case, a repo with just the split files but no maintained history clones
in ~12s in the above test setup. It also brings the (bare) repo down from 3,3GB
to 189MB. So the issue is really the data/CVE/list file.

That said, data/DSA/list is 14575 lines. That seems to not bother git too much
yet. Still if things get re-structured, this file may be worth a look, too.

To me the most reasonable path forward unfortunately looks like start a new repo
for 2019+ and "just" import the split files or single-record files as mentioned
by pabs but not the git/svn/cvs history. The old repo would - of course - stay
around but frozen at a deadline.

Corsac also mentioned on IRC that the repo could be hosted outside of Gitlab.
That would reduce the pressure for some time.
But cgit and other git frontends (as well as 

Bug#907308: Bug depends on glibc version

[Daniel Lange]

as discussed with jwilk on irc:

This bug depends on the (g)libc version:

* stock Stretch is unaffected (libc6-2.24-11+deb9u3)
* Stretch with glibc from unstable (libc6-2.27-5) is affected
* stock Ubuntu 18.04.1 (libc6-2.27-3ubuntu1) is affected

Bug#876087: xscreensaver: source-less and unlicensed code at hacks/images/m6502/dmsc.asm

[Daniel Lange]

Hi Daniel,

thank you very much.

All the best,
Daniel

Bug#876087: Source code and license of dmsc.asm

[Daniel Lange]

Hi,

your acme code dmsc.asm is used in xscreensaver by Jamie Zawinski.

Apparently there have been issues filed before at very Freedom oriented
distributions that the file is not clearly licensed and the source code
is not shipped with it. E.g. at https://labs.parabola.nu/issues/131 .

This has boiled up to Debian now at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876087
and I think the obvious thing is to ask you:

Could you please release this file under a DFSG compatible license
(https://www.debian.org/legal/licenses/) and provide the source e.g. as
a comment to the bug report and/or via email to Jamie (CC)?

Thank you for your consideration!

Best regards,
Daniel

Bug#827304: unable to reproduce the issue on three AMD64 systems

[Daniel Lange]

Control: tags -1 unreproducible

I've tried to reproduce the issue for 48h on three AMD64 systems I've
happened to have around here temporarily with fresh Debian testing
installs. Got to use the opportunity. One of the systems was used for
heavy editing, the two other for occasional editing with lots of idle
times. The theme was set to oblivion on mousepad invocation as
instructed by the bug submitter.
There were no crashes and no uninstructed switch of themes.
The memory consumption was also bound and only changed when editing and
saving.

So this look like an issue that may have been specific to the affected
system. As there have been no confirmations in three months either,
I suggest reducing the bug severity.

Bug#834502: Suggest to remove this old version

[Daniel Lange]

tcpcopy 0.6.3 relies on ip_queue support that has been removed from the
Linux kernel >= 3.5.0.

tcpcopy 0.7.0+ knows about nfqueue for kernels >= 3.5.0.

The current version of tcpcopy is 1.0.0 released 03.08.2015.

I suggest removing this old version from testing / sid.
(It will be auto-removed due to the RC bug from Stretch.)

NB1: intercept and tcpcopy have been split into separate repositories
upstream.
NB2: Ubuntu dropped the version 0.6.3 for the raring release (13.04),
cf. LP: #1163290
(https://bugs.launchpad.net/ubuntu/+source/tcpcopy/+bug/1163290)

Bug#834502: Previous subject should have been: patch attached

[Daniel Lange]

sorry, see ^

The input word is guaranteed to be at most STRINGSIZE-1 in length. One of the
mangle operations involves duplicating the input word, resulting in a string
twice the length to be accommodated by both area variables.

Howard Guo  2016-08-17

diff -rupN 3/lib/rules.c 3-patched/lib/rules.c
--- 3/lib/rules.c	2016-08-16 14:16:24.033261876 +0200
+++ 3-patched/lib/rules.c	2016-08-17 13:57:14.485782894 +0200
@@ -434,9 +434,8 @@ Mangle(input, control)		/* returns a poi
 {
 int limit;
 register char *ptr;
-static char area[STRINGSIZE];
-char area2[STRINGSIZE];
-area[0] = '\0';
+static char area[STRINGSIZE * 2] = {0};
+char area2[STRINGSIZE * 2] = {0};
 strcpy(area, input);
 
 for (ptr = control; *ptr; ptr++)

Bug#834502: Proposed NMU, debdiff attached

[Daniel Lange]

Control: tags -1 + patch

The buffer overflow results from strings that are too short for a strcpy to 
always succeed.

Patch from 

 attached.

The input word is guaranteed to be at most STRINGSIZE-1 in length. One of the
mangle operations involves duplicating the input word, resulting in a string
twice the length to be accommodated by both area variables.

Howard Guo  2016-08-17

diff -rupN 3/lib/rules.c 3-patched/lib/rules.c
--- 3/lib/rules.c	2016-08-16 14:16:24.033261876 +0200
+++ 3-patched/lib/rules.c	2016-08-17 13:57:14.485782894 +0200
@@ -434,9 +434,8 @@ Mangle(input, control)		/* returns a poi
 {
 int limit;
 register char *ptr;
-static char area[STRINGSIZE];
-char area2[STRINGSIZE];
-area[0] = '\0';
+static char area[STRINGSIZE * 2] = {0};
+char area2[STRINGSIZE * 2] = {0};
 strcpy(area, input);
 
 for (ptr = control; *ptr; ptr++)

Bug#833655: Proposed NMU, debdiff attached

[Daniel Lange]

Control: tags -1 + patch

Cherrypicked the METAURL change from
<https://github.com/keesL/metar/commit/6ce5ef8960d9e669a7583d215f3b222f7f272aa7>

diff -Nru metar-20061030.1/debian/changelog metar-20061030.1/debian/changelog
--- metar-20061030.1/debian/changelog   2016-07-16 13:11:56.0 +0200
+++ metar-20061030.1/debian/changelog   2016-08-22 16:26:06.0 +0200
@@ -1,3 +1,10 @@
+metar (20061030.1-2.2) unstable; urgency=medium
+
+  * Non-maintainer upload.
+  * Import patch for new METAR URL from Kees Leune. Closes: #833655
+
+ -- Daniel Lange <dl@usrlocal.de>  Mon, 22 Aug 2016 16:25:57 +0200
+
 metar (20061030.1-2.1) unstable; urgency=medium
 
   * Non-maintainer upload.
diff -Nru metar-20061030.1/debian/patches/fix-metarurl 
metar-20061030.1/debian/patches/fix-metarurl
--- metar-20061030.1/debian/patches/fix-metarurl1970-01-01 
01:00:00.0 +0100
+++ metar-20061030.1/debian/patches/fix-metarurl2016-08-22 
16:25:14.0 +0200
@@ -0,0 +1,24 @@
+From 6ce5ef8960d9e669a7583d215f3b222f7f272aa7 Mon Sep 17 00:00:00 2001
+From: keesL <k...@leune.org>
+Date: Mon, 8 Aug 2016 09:51:25 -0400
+Subject: [PATCH] Updated default URL from which to get metars. NOTE. This also
+ updated the protocol from FTP to HTTP
+Origin: upstream, 
https://github.com/keesL/metar/commit/6ce5ef8960d9e669a7583d215f3b222f7f272aa7
+
+---
+ src/metar.h | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/metar.h b/src/metar.h
+index c55d92d..b2537fc 100644
+--- a/src/metar.h
 b/src/metar.h
+@@ -24,7 +24,7 @@
+ #define  METAR_MAXSIZE 512
+ 
+ /* where to fetch reports */
+-#define  METARURL 
"http://weather.noaa.gov/pub/data/observations/metar/stations;
++#define  METARURL "http://tgftp.nws.noaa.gov/data/observations/metar/stations;
+ 
+ /* clouds */
+ typedef struct {
diff -Nru metar-20061030.1/debian/patches/series 
metar-20061030.1/debian/patches/series
--- metar-20061030.1/debian/patches/series  2016-07-16 13:03:57.0 
+0200
+++ metar-20061030.1/debian/patches/series  2016-08-22 16:25:31.0 
+0200
@@ -1 +1,2 @@
 fix-pod-errors
+fix-metarurl

Bug#820331: cronic: uses very predictable temporary files

[Daniel Lange]

Am 10.04.2016 18:46, schrieb Salvatore Bonaccorso:
CVE-2016-3992 has been assigned for this issue. Can you forward this 
to upstream and as well include the CVE id reference in 
debian/changelog when fixing this issue? 
Upstream has already fixed yesterday and I packaged the v3 for Debian 
this morning.

It's just waiting for Graham to upload. I'm a mere mortal, I can't :).