Bug#1040914: dev-ref: update best practices around security (Re: Securing Debian Manual too old?)

[Holger Levsen]

package: developers-reference
x-debbugs-cc: debian-security@lists.debian.org

hi,

On Tue, Jul 11, 2023 at 10:46:20PM +0200, Moritz Mühlenhoff wrote:
> > I found the Securing Debian Manual
> > (https://www.debian.org/doc/manuals/securing-debian-manual/index.en.html).
> > This version is from 2017.
> 
> This document is in fact too outdated and not in a shape we should
> prominently present it on the Debian website, thanks for flagging it.
> It even predates systemd and no mention of it at all...
> 
> Can you please "reportbug www.debian.org" asking to remove it from the
> website?

https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.en.html#best-practices-around-security

currently contains this text:



Best practices around security


A set of suggestions and links to other reference documents around
security aspects for packaging can be found at the `Developer's Best
Practices for OS Security chapter inside the Securing Debian Manual
`__.



and unsure what to do now, as I'd like to keep the anchor and chapter, so
just dropping this would be wrong. Help welcome.

> It's also packaged as src:harden-doc and probably stick around in
> case someone wants to improve it going forward.

I'm not even sure this is useful to keep around. :/


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Just today, over 800 women will have died due to preventable pregnancy and
birth complications, over 130 due to femicide.
https://www.who.int/news-room/fact-sheets/detail/maternal-mortality
https://en.wikipedia.org/wiki/Femicide#Worldwide


signature.asc
Description: PGP signature

Re: CVE-2017-5715

[Holger Levsen]

On Wed, Mar 30, 2022 at 09:36:58AM +0200, Sylvestre Ledru wrote:
> Le 30/03/2022 à 07:07, Salvatore Bonaccorso a écrit :
> > Sylvestre and Holger, would you have time to include the bugfix as
> > well in the future bullseye point release?
> Sure, should be easy.
> Is there a timeline?

as the last point release was last weekend the next one will probably
happen in around two months.

that said, one can file an SRM bug now and do the upload now as well too. :)


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Nach wieviel Einzelfällen wird ein Einzelfall zum Normalfall?
(Jan Böhmermann)


signature.asc
Description: PGP signature

thank *you*, team@security.d.o! (was Re: [SECURITY] [DSA 5000-1] openjdk-11 security update)

[Holger Levsen]

hey hey, hear hear!

On Mon, Nov 01, 2021 at 07:44:34PM +, Moritz Muehlenhoff wrote:
> -
> Debian Security Advisory DSA-5000-1   secur...@debian.org

WHHO!

that's *something* to *celebrate*!!1 Very many thanks to the whole Debian
security team, past and present, and to everyone contributing! You rock!
A lot! 5 whooping thousand (counted) times so far!

Thank you very much once more, and not enough, not even close.

!


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

It's climate crime, not climate change.


signature.asc
Description: PGP signature

Re: sources.list 4 bullseye-security

[Holger Levsen]

On Sun, Jun 27, 2021 at 04:52:26PM -0400, Boyuan Yang wrote:
> Besides, I believe end users are not supposed to know deb-src line for
> security repos.

sure, they do! and of course we provide source for our security updates!

> Adding such info provides zero benefit except for confusing
> users.

surely not all users compile software, but some certainly do. I do.


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

„Faschisten hören niemals auf, Faschisten zu sein
Man diskutiert mit ihnen nicht, hat die Geschichte gezeigt“...


signature.asc
Description: PGP signature

Bug#989307: DSA-4923-1: upgrading libwebkit2gtk-4.0-37 on buster pulls in xdg-desktop-portal

[Holger Levsen]

Package: libwebkit2gtk-4.0-37
Version: 2.32.1-1~deb10u1
Severity: normal

Dear Maintainer,

from #debian-security today, Salvatore asked me to file this as a bug.

< h01ger> DSA 4923 causes xdg-desktop-portal(-gtk) to be installed here, much 
to my surprise and unhappyness
< h01ger> its a recommends, so i can apt remove it, but still...
< h01ger> https://paste.debian.net/1199471/
(which has this content)

Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
The following NEW packages will be installed:
   libpipewire-0.2-1 (0.2.5-1)
   xdg-desktop-portal (1.2.0-1)
   xdg-desktop-portal-gtk (1.2.0-1)
The following packages will be upgraded:
   libjavascriptcoregtk-4.0-18 (2.30.6-1~deb10u1 => 2.32.1-1~deb10u1)
   libwebkit2gtk-4.0-37 (2.30.6-1~deb10u1 => 2.32.1-1~deb10u1)
2 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 20.1 MB of archives.
After this operation, 5,376 kB of additional disk space will be used.
Do you want to continue? [Y/n] 

< carnil> h01ger: thanks forwarded to alberto
* h01ger still busy cleaning systems
< h01ger> carnil: thanks!
< carnil> h01ger: the problem which is to be solved with it is apparently 
https://bugzilla.redhat.com/show_bug.cgi?id=1845743 (according to berto)
< h01ger> carnil: seems like. i've no flatpak and no snap and i dont expect to 
gain a dbus service granting privileges on a buster security update. (i've also 
seen it on bullseye upgrades but given that bullseye is not stable yet i wont 
complain here :)
< h01ger> carnil: do you think it would be useful if i'd file a bug about this 
issue?
< carnil> h01ger: at least the maintainer could comment himself on it, and 
explain on why the recommends, and maybe discussion can lead to that change is 
not suitable for the DSA, and we can drop it in the next upload.
< h01ger> carnil: was that a yes?
< carnil> h01ger: yes
< h01ger> carnil: ok. i'll include some lines from you here...
< carnil> h01ger: yes. I have no proplem if you mention I suggested to fill a 
but to ask to clarify the issue. But note I was only inbetween here.


Personally I think a DSA fixing this would be nice.

-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

"Climate change" is an euphenism. "Global warming" as well.


signature.asc
Description: PGP signature

Re: "Version less than 0.0" in OVAL definitions

[Holger Levsen]

On Sun, May 16, 2021 at 05:21:50PM +0300, Serkan Özkan wrote:
> We are using Debian OVAL definitions but there are many tests, and states,
> that test for dpkg versions being less than 0.0 which is impossible in
> practice (right?).

no, it's possible:

0~1 is a valid version. It's smaller than zero, yet it's not a negative
number.

It's usually used for versions like 1.0~0alpha1-1 to allow the next
version to be 1.0-1... but 0~1 is a legal and valid version too.


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

I'm looking forward to Corona being a beer again and Donald a duck.


signature.asc
Description: PGP signature

fun with mailinglists (was Re: Is chromium updated?)

[Holger Levsen]

On Fri, Nov 13, 2020 at 12:06:50PM +0200, Georgi Guninski wrote:
> On Fri, Nov 13, 2020 at 10:21 AM Pavlos Ponos  wrote:
> > BUT we should not forget to say a THANK YOU to these guys which give their 
> > best in order all of us to use this OS for free ;-)
> I believe I am debian contributor too, search in google for:
> "georgi guninski" site:debian.org
 
you seem to be a very funny person, less than 3h ago you said in 
Message-ID: 
Debian was not responding to this thread and now you are saying you
are Debian too! :)))


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁   holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄

Moral, truth, long term- and holistic thinking seem to mean nothing to us. The
emperors are naked. Every single one. It turns out our whole society is just
one big nudist party. (Greta Thunberg about the world reacting to the corona
crisis but not reacting appropriatly to the climate crisis.)


signature.asc
Description: PGP signature

how to deal with widely used packages unsuitable for stable (was Re: [Git][security-tracker-team/security-tracker][master] Add radare2 to dla-needed.txt with comments.)

[Holger Levsen]

hi,

(this started as a discussion whether to update radare2 in (old)stable
and has since then evolved into a discussion about the problem
summarized well by Raphael.)

On Thu, Aug 29, 2019 at 01:48:14PM +0200, Raphael Hertzog wrote:
> On Thu, 29 Aug 2019, Moritz Mühlenhoff wrote:
> > The upstream link makes it sound as if they are one of those upstreams
> > which reject the idea of distributions shipping an older release to
> > a stable distro. For a tool like radare2 that seems fair enough, so
> > how about simply excluding it from stable releases (and retroactively
> > drop it from Buster/Stretch in the forthcoming point releases)?
> 
> 
> While I have no problem in getting it out of stable release, it is
> important that we are able to provide backports so the package must
> stay in Debian testing. 
> 
> 
> 
> Also radare2 is a package that we care about in Kali and we are based
> on Debian testing so we would prefer if it could continue to be there.
> 
> 
> In general, we (Debian) don't have a good answer to this problem and
> virtualbox is clearly a bad precedent. We really need to find a solution
> to this in concertation with the release managers.

so I've added them to this thread.

youtube-dl is in the same boat...


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: [Git][security-tracker-team/security-tracker][master] Add radare2 to dla-needed.txt with comments.

[Holger Levsen]

On Fri, Aug 16, 2019 at 08:11:58PM +, Markus Koschany wrote:
> Markus Koschany pushed to branch master at Debian Security Tracker / 
> security-tracker
> 
> Commits:
> bc35662f by Markus Koschany at 2019-08-16T20:11:47Z
> Add radare2 to dla-needed.txt with comments.
> 
> - - - - -
> 1 changed file:
> - data/dla-needed.txt
> +radare2
> +  NOTE: 20190816: Affected by CVE-2019-14745. Vulnerable code is in
> +  NOTE: libr/core/bin.c. Many no-dsa issues in Jessie and Stretch. Should we
> +  NOTE: continue the current approach, update to a newer upstream version or 
> mark
> +  NOTE: radare2 as unsupported? Also note that there is a r2-pwnDebian 
> challenge...
> +  NOTE: https://bananamafia.dev/post/r2-pwndebian/ (apo)

I'd be in favor of marking radare2 as unsupported, probably even for stable,
but definitly for oldstable and older.

I'd be happy to do these changes in src:debian-security-tracker and
uploading this to sid.


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: rssh security update breaks rsync via Synology's "hyper backup"

[Holger Levsen]

Hi Roman,

the security team is not responsible for Debian LTS, I've thus added 
debian-lts@lists.d.o to the mail recipients, so that they become aware
of your issue.

On Thu, Feb 14, 2019 at 06:06:34PM +0100, Roman Medina-Heigl Hernandez wrote:
> Hi security-fellows,
> 
> I applied recent rssh security updates to Debian 8 (jessie) and I
> noticed that it breaks Synology's "Hyper backup" tool (with rsync method).
> 
> The relevant log lines at my Debian server:
> 
> Feb 10 03:28:21 roman rssh[19985]: cmd 'rsync' approved
> Feb 10 03:28:21 roman rssh[19985]: insecure rsync options in rsync
> command line!
> Feb 10 03:28:21 roman rssh[19985]: user synology attempted to execute
> forbidden commands
> Feb 10 03:28:21 roman rssh[19985]: command: rsync --server --daemon .
> 
> Is it really unsafe to issue a "rsync --server --daemon ." command so it
> deserves to be blocked?`
> 
> 
> PS: OS info:
> 
> root@roman:~# cat /etc/debian_version
> 8.11
> root@roman:~# dpkg -l rssh   
> Deseado=desconocido(U)/Instalar/eliminaR/Purgar/retener(H)
> |
> Estado=No/Inst/ficheros-Conf/desempaqUetado/medio-conF/medio-inst(H)/espera-disparo(W)/pendienTe-disparo
> |/ Err?=(ninguno)/requiere-Reinst (Estado,Err: mayúsc.=malo)
> ||/ Nombre    Versión
> Arquitectura    Descripción
> +++-=-===-===-
> ii  rssh  2.3.4-4+deb8u2 
> amd64   Restricted shell allowing scp, sftp, cvs, svn,
> rsync or rdist
> 
> PS2: I'm not suscribed to LTS-list, but I guess the problem may be both
> in stable and oldstable versions.
> 
> Cheers,
> 
> -Román
> 

-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C

In Europe there are people prosecuted by courts because they saved other people
from drowning in the  Mediterranean Sea.  That is almost as absurd  as if there
were people being prosecuted because they save humans from drowning in the sea.


signature.asc
Description: PGP signature

Bug#922247: Bug#859122: about 500 DLAs missing from the website

[Holger Levsen]

Hi Salvatore,

On Tue, Feb 12, 2019 at 08:13:18AM +0100, Salvatore Bonaccorso wrote:
> I have the attached patch commited in a local branch, but want first
> to confirm is this the final intended URL to reach the DLAs?
> -return 
> url.absolute("https://www.debian.org/security/%d/dla-%d;
> +return 
> url.absolute("https://www.debian.org/security/lts/%d/dla-%d;


the DLAs are visible on https://www.debian.org/lts/security/ now and I
believe thats a good url ment to stay ;)


-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Bug#922247: security-tracker: please use new urlpath for DLAs on www.d.o

[Holger Levsen]

package: security-tracker
x-debbugs-cc: debian-...@lists.debian.org

Hi,

this is a bug to track fixing this small glitch in the new
www.debian.org/lts/security/ area:

On Mon, Feb 11, 2019 at 04:26:38PM -0500, Antoine Beaupré wrote:
> >> * Adaptation in the security tracker so the new URL paths are used from
> >> now on is also needed.
> > right. shall we file a bug to not forget this?
> Sure, please do.

done. Salvatore also prepared a patch for this.


-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: Should easter eggs be disabled in Debian's php packages?

[Holger Levsen]

On Fri, Jan 18, 2019 at 01:58:12PM +0800, Paul Wise wrote:
> > To answer my own question, after PHP 5.5 the easter egg was removed already.
> So the issue would only be present in wheezy. I guess the ELTS folks
> might like to disable them.

I don't think the behaviour of php should be changed at this time,
unless this is really security relevant, which AFAICS has not been
demonstrated yet.

(Proof of the contrary welcome! :)


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Bug#908678: Testing the filter-branch scripts

[Holger Levsen]

On Wed, Nov 14, 2018 at 07:45:59PM +0100, Moritz Muehlenhoff wrote:
> Nearly all the tasks of actually editing the data require a look at the 
> complete
> data, e.g. to check whether something was tracked before, whether there's an 
> ITP
> for something, whether something was tracked as NFU in the past and lots more.

according to git log, the data goes back to 2004. Do you really need all
those 15 years of history or could we maybe make a yearly split for
(now) the first 10 years and have the last 5 years in "one"?

And then when we move into 2019 we would move 2014 to the then 11 first
years and so on... same in 2020 with 2015 then...

IMHO we should do something, else dealing with security-tracker.git will be
even more cumbersome in 5 or 10 years ahead.


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: Gaps in security coverage?

[Holger Levsen]

On Tue, Nov 06, 2018 at 07:08:20PM +0800, Paul Wise wrote:
> Bug#908678: security-tracker - Breaks salsa.d.o
 
thank you.


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: DLA link is broken

[Holger Levsen]

On Tue, Nov 06, 2018 at 07:45:24AM +0100, Salvatore Bonaccorso wrote:
> >  DLA link is broken.
> >  e.g. https://security-tracker.debian.org/tracker/DLA-1445-1 page
> >  "SourceDebian LTS" points to 
> > https://www.debian.org/security/2018/dla-1445
> >  but there's no such page.
> Cf. #762255 and related bugs which added support for having the DLA's
> included both in security-tracker source field and on the website.

that bug and its clone are all closed.

> Though this needs volunteers to actually import and translate the
> DLAs.

import to where?

(i'll leave out translations for now...)


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: Gaps in security coverage?

[Holger Levsen]

On Tue, Nov 06, 2018 at 02:42:59PM +0800, Paul Wise wrote:
> Also, a much more important task is restructuring the git repo so that
> it doesn't cause responsiveness and resource usage issues with salsa.

is there a bug or wiki page describing the issues/requirements for that and
what has been tried / the status?

(I just cloned the tracker yesterday and could see the problem 'live'..)


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: Bug#907723: link package versions on security-tracker to source packages

[Holger Levsen]

On Sat, Sep 01, 2018 at 12:43:58PM +0800, Paul Wise wrote:
> > So, I always go to [1] with my web browser, copy the URL of the .dsc file
> > and then dget that .dsc file.
> This misses out verifying apt signatures.

the .dsc file is signed and dget verifies it.


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: powerpc update for amd64

[Holger Levsen]

On Sun, Mar 04, 2018 at 04:07:14PM +0100, SZÉPE Viktor wrote:
> Why should one using an amd64 hardware update its kernel/reboot when changes
> are only for powerpc?
 
you should not. (or maybe you should so your monitoring will not
complain about running an outdated kernel.)

however, because the same linux kernel source package is used to build
the linux kernel binary packages for all archs, the amd64 packages are
also updated when there are no changes relevant to amd64.


-- 
cheers,
Holger


signature.asc
Description: PGP signature

Re: retpoline-enabled GCC build for jessie

[Holger Levsen]

On Sat, Feb 17, 2018 at 02:35:22PM +0100, Moritz Mühlenhoff wrote:
> The update for gcc-4.9 has just been released.
> Test packages for gcc-6/stretch are now available at 
> https://people.debian.org/~jmm/gcc6/
 
Thanks for your work on this, Moritz.

I have a stupid/uninformed question: is this gcc only useful for
rebuilding the kernel or would it "in theory" (and practice) be better
to rebuild everything with it? (of course the latter is probably not really
practical for Debian, but others could do it more easily.)


-- 
cheers,
Holger


signature.asc
Description: PGP signature

Re: Plannings for secure-testing repository migration to git

[Holger Levsen]

On Tue, Dec 26, 2017 at 03:29:08PM +0100, Salvatore Bonaccorso wrote:
> FTR, so now that the beta for salsa.d.o has been announced I started
> to look on what further is needed and recorded further findings in
> TODO.gitmigration.

\o/

thanks for doing this work!

> [...] Personally I still would have appreciated a commit mailinglist,
> since at least Moritz' and mine's review work was done that way so
> far. But I realize no option in this direction arised.
 
I think a security-tracker-commit maillinglist does make sense on
lists.debian.org as the security tracker is a central part of Debian's
(security) infrastructure. I'd suggest to talk to listmasters about this
(unless you already done this, obviously).


-- 
cheers,
Holger


signature.asc
Description: PGP signature

Re: Is packages build without verifying the source package signatures?

[Holger Levsen]

On Sun, Dec 03, 2017 at 01:11:50PM +0100, Bastian Blank wrote:
> It would still only need to compromise one machine: The one from where
> the keys are handled and distributed.

I rest my case. I'd secure the front door even if the side door (atm
still) can be compromised easy.


-- 
cheers,
Holger


signature.asc
Description: PGP signature

Re: Is packages build without verifying the source package signatures?

[Holger Levsen]

On Sun, Dec 03, 2017 at 12:05:51PM +0100, Bastian Blank wrote:
> > in practice, this also has obvious flaws.
> Please elaborate.

for a start: one only needs to compromise one machine instead of many...

> >   what's the technical reason
> > the buildds are not checking the signatures?
> Unavailability of the keys.  Key may have been expired between upload
> and build attempt.

I'm not sure this is an advantage then... or rather: I'd rather see a
requirement that keys used for signing are valid for at least another
year after the upload.


-- 
cheers,
Holger


signature.asc
Description: PGP signature

Re: Is packages build without verifying the source package signatures?

[Holger Levsen]

On Sun, Dec 03, 2017 at 12:38:24PM +0800, Paul Wise wrote:
> The Debian buildds only do the first verification (due to all Debian
> package uploader keys not being installed) but the Debian archive
> verifies that all uploads match a known developer key before passing
> packages to the buildds. So in practice, both verifications are
> happening, but not in the same place.
 
in practice, this also has obvious flaws. what's the technical reason
the buildds are not checking the signatures?


-- 
cheers,
Holger


signature.asc
Description: PGP signature

Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups

[Holger Levsen]

On Sat, May 13, 2017 at 10:48:18PM +0200, Aurelien Jarno wrote:
> The above change should now be deployed on most jessie based buildds,
> it's only missing on the buildds that are currently down.

cool, thank you!


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: [buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups

[Holger Levsen]

On Sat, May 13, 2017 at 05:52:04PM +0200, Mattia Rizzolo wrote:
> On Sat, May 13, 2017 at 03:44:57PM +0100, Chris Lamb wrote:
> >  a) Has anything changed in the meantime?
> 
> Yes: sbuild stopped repeating the changelog time taking it from the last
> entry, and will instead generate a new timestamp based on the current
> time:
> 
>   * For binNMUs, instead of copying the timestamp of the last changelog entry,
> generate a new one (closes: #843773)
> 
> In version 0.73.0-1.

this is correct, but AFAIK this hasnt been deployed on the buildd yet.
I'd be glad to be corrected about this…
 
> >  b) Will this affect stretch? If so, what do we need to do now?
> IMHO, nothing.

we might need to reschedule some packages…


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Bug#761945: fixing links for DLAs in the security tracker

[Holger Levsen]

On Wed, Mar 29, 2017 at 07:29:06AM +0200, Salvatore Bonaccorso wrote:
> The security-tracker side of this has been implemented now, Paul Wise
> did the corresponding work.
 
cool! thanks Paul!


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Some Debian package upgrades are corrupting rsync "quick check" backups

[Holger Levsen]

On Mon, Jan 30, 2017 at 02:47:45PM +0100, Johannes Schauer wrote:
> > (the sbuild maintainer reads the above list which has been cc:ed so he
> > should be able to comment…)
> 
> You were talking about buildd-tools-de...@lists.alioth.debian.org

yes

> You forgot to CC that one (I understood that was your intention) but I'm also
> still subscribed to reproducible-builds@l.a.d.o. :)

no, I didnt forget to cc: the list as I didnt know it's exact name and I
was too lazy to look for it as I knew you read the reproduible list ;)

[nice explaination deleted, nothing to add here, except thanks for
explaining.]

> > so, two questions:
> > 
> > a.) has been fixed, so that no new occurrances of this problem will occur?
> Hopefully. I welcome reports that show the contrary.

you are refering to "fixed in sbuild" here, while I ment (but didnt excplitly
say…) "fixed in the buildd network"…

> > b.) if thats the case, shall we scan all packages in sid for files which
> > have the same timestamp+filename but different checksums and ask for
> > binNMUs of those packages?
> The version of sbuild used on the buildds probably doesn't bump the timestamp
> yet. At least the binNMU-ed packages on my system share the binNMU changelog
> timestamp with the timestamp of the last source upload. :)

so it seems we should 

a.) get this fix deployed on all the buildds. (how?)
b.) do the b.) above… (because AIUI this issue atm is only be noticed
by those running unstable or stretch, while once stretch is released many
more people will notice it…)


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Some Debian package upgrades are corrupting rsync "quick check" backups

[Holger Levsen]

On Mon, Jan 30, 2017 at 01:10:12PM +0100, Mattia Rizzolo wrote:
> > Would reproducible-bui...@lists.alioth.debian.org be the correct mailing
> > list to discuss this?
 
the debian-buildd list or a bug against sbuild might be more
appropriate…

(the sbuild maintainer reads the above list which has been cc:ed so he
should be able to comment…)

> Not really, because that has been done in sbuild since long before the
> reproducible builds project became active: 0.62.2-1, Tue, 05 Apr 2011:
> - Improve binNMU handling to permit binNMUs for multiarch packages
>   (Closes: #620112).  Currently, binary NMUs use the current date
>   in the new changelog entry, but co-installable packages require
>   an identical changelog.  To avoid this, take the date from the
>   previous changelog entry to ensure the same date for all binNMUs.
>   Thanks to Anders Kaseorg for this patch.
> 
> And, incidentally, this has been kind of reverted in 0.73.0-1 (Sat, 24
> Dec 2016) after a fairly long and annoying discussion in debian-devel:
>   * For binNMUs, instead of copying the timestamp of the last changelog entry,
> generate a new one (closes: #843773)

so, two questions: 

a.) has been fixed, so that no new occurrances of this problem will occur?
b.) if thats the case, shall we scan all packages in sid for files which
have the same timestamp+filename but different checksums and ask for
binNMUs of those packages?

thinking about b.) debian-release@l.d.o might be the right list for
this…


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Some Debian package upgrades are corrupting rsync "quick check" backups

[Holger Levsen]

On Sat, Jan 28, 2017 at 03:04:56PM +0100, Daniel Reichelt wrote:
> I highly suspect this stems from packages' rules files supporting
> reproducible builds.

I rather think this is due to binNMUs not modifying debian/changelog…
(in the source package while it's modified in the binary packages…)


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Handling of "malware" in Debian

[Holger Levsen]

On Wed, Nov 09, 2016 at 07:14:45PM +0100, W. Martin Borgert wrote:
> If users of testing or unstable have the malware installed now and
> the package gets removed from the archive, users are left with the
> malware, right?

yes
 
> That's why I thought about uploading an empty package to unstable,

yes, of course.

> it should be released with stretch, but can be safely removed later.

i'm not sure about the releasing with stretch part. Maybe it would be
better to have the updated, empty package in stretch in 5plusX days and
then remove it before the release, say on January 1st.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Handling of "malware" in Debian

[Holger Levsen]

On Wed, Nov 09, 2016 at 04:17:58PM +0100, W. Martin Borgert wrote:
> Would NEWS.Debian be sufficient?

I think so. And I also think this should be done.

and, who's gonna file the RM bug for unstable?


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: flashplugin-nonfree and latest Flash security updates

[Holger Levsen]

On Thu, Aug 04, 2016 at 02:14:55AM +, Nick Boyce wrote:
> > Just don't use that crap. With the amount of zero days in Flash
> > you're subject to serious vulnerabilities even with an up-to-date
> > plugin.
> [...]  Also I
> believe there are quite a few corporate intranet use-cases that *depend*
> on Flash for corporate web-apps (at least according to traffic on the
> Enterprise Firefox list).

plus there are a lot of school + education related websites & other
material, including exams, which need flash… I estimate it will take
some years to weed those out.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: flash plugin from ubuntu (was: flashplugin-nonfree and latest Flash security updates)

[Holger Levsen]

On Wed, Aug 03, 2016 at 10:46:33PM +0200, Stefan Fritsch wrote:
> Maybe the flashplugin-nonfree package should even be replaced by a package 
> that 
> installs the ubuntu archive signing key, sets up the sources.list line, and 
> tweaks the unattended-updates config to allow automatic updates from that 
> repo. 

please, no.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Call for testing: upcoming wordpress security update

[Holger Levsen]

On Tue, Aug 02, 2016 at 04:37:31PM +0200, Jakub Wilk wrote:
> Wiki is world-writable. It's safe to assume that everything there is
> nonsense unless proven otherwise.
 
It's also safe to assume that we'll al die one day, though that's also
not very helpful.

A useful first step to assess the qualilty of the information on any given page
on wiki.d.o is usually to look at the page history and see who edited
it.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: httpoxy efforts?

[Holger Levsen]

Hi Christoph,

your email doesnt mention whether you searched the BTS for relevant bugs
about these issues. Have you?

And if there are no bugs filed yet, someone should file bugs.

:-)


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: the frustrated administrivia and misdirection hose lacks any abatement visible to mortals

[Holger Levsen]

Hi Drake,

On Tue, May 24, 2016 at 01:32:08PM +0800, Paul Wise wrote:
> > Lacking any obvious way to talk to the security team without potentially 
> > making my
> > message look more urgent than it was, I leave it to whoever else can 
> > navigate the
> > Debian social structure to take it up in the most appropriate manner.  I've 
> > absolutely run
> > out of nerves for having to clear this garbage out of my mailpile, so I'm 
> > done here.
> 
> Two of the security team members responded to the bug report:
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821113#25
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821113#20
> 
> So the only thing that needs doing now is for the listmasters to
> implement the suggestions.

after I read all the quoted above, I was sceptical when I opened
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821113 but after
having read that too, I just want to say "thanks" and "kudos" to you.

As explained by Moritz the current behavior is mostly historic and often it's
not easy to change such historic things. It seems to me that you managed
to make a good+doable proposal *and* put it at right place(!) so I'm looking
forward to an implementation of your proposal now.

Yay!


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Which Debian packages leak information to the network?

[Holger Levsen]

On Wed, May 18, 2016 at 06:33:52PM +0200, Jakub Wilk wrote:
> Could you explain how any of these tools leak any information "without a
> user's consent/expectation"?

gnome-calculator contacts a web page/service with currency exchange
information *on every start*, I think that's a good example of the kind
of programs Patrick is looking for.


-- 
cheers,
Holger


signature.asc
Description: Digital signature

Re: Should Debian ask for a CPE when a CVE in Debian is found?

[Holger Levsen]

Hi,

On Samstag, 13. Februar 2016, Paul Wise wrote:
> On Sat, Feb 13, 2016 at 2:51 AM, Wheeler, David A wrote:
> > Should Debian's security team ask for a Common Platform Enumeration (CPE)
> > id when a related CVE is found/reported fixed?
> 
> The debian-security list is a general Debian security discussion list
> rather than a contact point for the Debian security team.

yeah, exactly, that's why I suggested David to discuss this on this list. 

> If you wish
> to contact the Debian security team, please use secur...@debian.org.

That is not an address suited for public discussion (it aint public and there 
is no public archive), so your suggestion aint much helpful here.

Debian usually works in the open, as I understand it secur...@debian.org is 
for telling stuff to the Security team which aint open yet.

If debian-security@lists.debian.org should not be used to discuss security 
topics related to Debian (with and without the security team) this should be 
clarified, though I doubt this is the case.


Now if only someone could reply to the original question at hand! ;-)


cheers,
Holger



signature.asc
Description: This is a digitally signed message part.

Re: Security support incomplete? (was: Re: [SECURITY] [DSA 3455-1] curl security update)

[Holger Levsen]

Hi Wolfgang,

On Dienstag, 2. Februar 2016, Wolfgang Jeltsch wrote:
>   • Where does the tracker talk about security policies? (I actually
> doubt that such information is in the tracker at all.)

That's out of scope for the tracker indeed, however right now I dont know 
where to find such policies.

>   • Where is a list of unfixed security issues?

https://security-tracker.debian.org/tracker/ links to filters for the 
different suites, eg "Vulnerable packages in the stable suite" points to 
https://security-tracker.debian.org/tracker/status/release/stable where you 
can tune your view.

So https://security-
tracker.debian.org/tracker/status/release/stable?filter=1=high_urgency=medium_urgency=low_urgency=unimportant_urgency=unassigned_urgency=undetermined_issues=nodsa
 
is probably the URL which will show you the highest number of security issues 
in stable ;)
 
> URLs would be highly appreciated.

not directly answering your questions, but maybe still useful:

http://security-team.debian.org/security_tracker.html


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: [SECURITY] [DSA 3448-1] linux security update

[Holger Levsen]

Hi,

On Mittwoch, 20. Januar 2016, Bjoern Nyjorden wrote:
> Most appreciated.  So, just to confirm; my take away on this is:
> 
>   * 1. "Wheezy" Linux kernels are NOT AFFECTED.
> 
>   * 2. "Wheezy" & "Jessie" BACKPORTS Linux kernels are VUNERABLE.
> 
> If I have understood correctly?

yes!


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: [SECURITY] [DSA 3448-1] linux security update

[Holger Levsen]

Hi Bjoern (bcc:ed),

On Mittwoch, 20. Januar 2016, Bjoern Nyjorden wrote:
> Are the "Wheezy" Linux kernels affected as well, or are they currently
> okay as far as you know?

on debian-backports@l.d.o Ben wrote:

> [...]  It's fixed in jessie and sid,
> and doesn't affect anything older.  {wheezy,jessie}-backports will be
> fixed soon.

Thanks Ben!


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: Downloading all information in JSON format

[Holger Levsen]

Hi Grant,

On Donnerstag, 7. Januar 2016, Grant Murphy wrote:
> I'm trying to build a tool that monitors security issues across a
> number of different sources. One of which was the Debian security
> tracker.

cool!
 
> I had hoped to periodically poll this url:
> https://security-tracker.debian.org/tracker/data/json for changes and
> update my local copy as required. After a bit of looking I was
> surprised to see that:
> 
>   - The download size is ~30mb (Acceptable if the below measures had been
> taken) 
> - The file has not had whitespace removed (will reduce file size by
> about 5mb) 

that sounds like a useful addition now. IIRC it was done on purpose to make 
the json more readable during development of it…

> - The file is not compressed (gzip will reduce __total__ file
> size to ~2mb) 

iirc it's compressed by apache on the fly…

> - It is generated per request

true + currently by design… (not saying this cannot be changed…)

> - The 'If-Modified-Since' header is not honored so the complete file
> will need to be re-downloaded every time to check if it has changed.
> 
> 
> So here are my questions:
> 
>   - Is there a better way to access this information than periodically
> polling this URL

no

(well you could poll the same data from svn instead…)

>   - Can we at the very least get some form of compressed version of this
> data? - If nobody has time to fix this how can I contribute?

svn.debian.org/svn/secure-testing has the code and in there it's 
bin/tracker_service.py which has a function page_json…


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#788362: security-tracker switched to ftp.de.d.o as httpredir returns 404 when it shouldnt

[Holger Levsen]

package: security-tracker
x-debbugs-cc: Raphael Geissert geiss...@debian.org

Hi Raphael,

httpredir as used for security-tracker.debian.org has some problems updating
some Packages files, _sometimes_. IOW: i've seen this working on my laptop,
but not when deployed on soler. The url exists... (or rather it does as 
Packages.gz, 
so it seems apt-update-file has some magic here too...) but it repeatedly
failed like this. (or similar for other targets too)

sectracker@soler:/srv/security-tracker.debian.org/website/secure-testing$ make 
update-oldstable
set -e ; for rel in wheezy ; do \
for archive in main contrib non-free ; do \
python bin/apt-update-file \

http://httpredir.debian.org/debian//dists/$rel/$archive/source/Sources \
data/packages/${rel}__${archive}_Sources ; \
done ; \
for arch in amd64 armel armhf i386 ia64 mips mipsel powerpc s390 s390x 
sparc kfreebsd-i386 kfreebsd-amd64 ; do \
  for archive in main contrib non-free ; do \
  python bin/apt-update-file \

http://httpredir.debian.org/debian//dists/$rel/$archive/binary-$arch/Packages \
data/packages/${rel}__${archive}_${arch}_Packages ; \
  done ; \
done ; \
done
error: in download of 
'http://httpredir.debian.org/debian//dists/wheezy/contrib/binary-sparc/Packages'
 to 'data/packages/wheezy__contrib_sparc_Packages':
Traceback (most recent call last):
  File bin/apt-update-file, line 29, in module
debian_support.updateFile(sys.argv[1], sys.argv[2])
  File 
/srv/security-tracker.debian.org/website/secure-testing/lib/python/debian_support.py,
 line 346, in updateFile
return downloadFile(remote, local)
  File 
/srv/security-tracker.debian.org/website/secure-testing/lib/python/debian_support.py,
 line 309, in downloadFile
lines = downloadGunzipLines(remote + '.gz')
  File 
/srv/security-tracker.debian.org/website/secure-testing/lib/python/debian_support.py,
 line 292, in downloadGunzipLines
data = urllib2.urlopen(remote, timeout=TIMEOUT)
  File /usr/lib/python2.7/urllib2.py, line 154, in urlopen
return opener.open(url, data, timeout)
  File /usr/lib/python2.7/urllib2.py, line 437, in open
response = meth(req, response)
  File /usr/lib/python2.7/urllib2.py, line 550, in http_response
'http', request, response, code, msg, hdrs)
  File /usr/lib/python2.7/urllib2.py, line 469, in error
result = self._call_chain(*args)
  File /usr/lib/python2.7/urllib2.py, line 409, in _call_chain
result = func(*args)
  File /usr/lib/python2.7/urllib2.py, line 656, in http_error_302
return self.parent.open(new, timeout=req.timeout)
  File /usr/lib/python2.7/urllib2.py, line 437, in open
response = meth(req, response)
  File /usr/lib/python2.7/urllib2.py, line 550, in http_response
'http', request, response, code, msg, hdrs)
  File /usr/lib/python2.7/urllib2.py, line 475, in error
return self._call_chain(*args)
  File /usr/lib/python2.7/urllib2.py, line 409, in _call_chain
result = func(*args)
  File /usr/lib/python2.7/urllib2.py, line 558, in http_error_default   
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found
Makefile:106: recipe for target 'update-oldstable' failed
make: *** [update-oldstable] Error 1

So for now I have switched the security-tracker to use ftp.de.debian.org again.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#783491: security-tracker: document what needs to be done on releases and other archive changes

[Holger Levsen]

Hi Salvatore,

On Dienstag, 5. Mai 2015, Salvatore Bonaccorso wrote:
 I think two more changes were actually needed to get the testing
 status view show the correct information: r34072 and 34073.

good catch, thanks!


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#783491: security-tracker: document what needs to be done on releases and other archive changes

[Holger Levsen]

package: security-tracker
severity: wishlist

Hi,

3fa31ab2a22a7e6db606899ca3ee6cb45a7884d1 / svnr33868 is commit showing what 
needs to be done on upgrades, specifically these files need to be updated:

Makefile# search for release-names
bin/tracker_data.py # search for release-names
bin/tracker_service.py  # search for release-names and -backports
lib/python/debian_support.py# search for release-names
lib/python/dist_config.py   # search for release-names
lib/python/security_db.py   # search for release-names

This should be documented in doc/README.releases. (And now I have this I'm 
pondering to just do and not file this bug... but that's 5 more minutes, 
so...)

And also rather obviously, this could (+should) be trimmed down by refactoring 
- or a rewrite ;-)


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#783491: security-tracker: document what needs to be done on releases and other archive changes

[Holger Levsen]

Hi Francesco,

On Montag, 27. April 2015, Francesco Poli wrote:
  3fa31ab2a22a7e6db606899ca3ee6cb45a7884d1 / svnr33868 is commit showing
 I am sorry to ask, but... is this commit supposed to be already live?

yes it is.
 
 I am asking since I still see a tracker situation inconsistent with the
 release of jessie.

I'd suggest to let this post-release situation resolve itself a bit (eg I also 
see inconsistencies on packages.qa.d.o and tracker.d.o), do some jessie 
installations or upgrades (+file bugs there if you encounter them), be happy 
about the release and look at again at the security-tracker in a day or two.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: security-tracker json deployed

[Holger Levsen]

Hi Raphael,

On Montag, 20. April 2015, Raphael Hertzog wrote:
 I just noticed that DLA/DSA end up referenced as security issues. See
 for example DLA-204-1 and DLA-27-1 assigned to file.

That's a bug, thanks for notifying. I will fix it soon, latest on saturday 
when I'll add oldoldstable support to the security-tracker.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#781029: include (dsa|dla)-needed in json output

[Holger Levsen]

x-debbugs-cc: 761859, hert...@debian.org
package: security-tracker
severity: wishlist

Hi,

On Montag, 16. März 2015, Raphael Hertzog wrote:
 Another nice thing to add in the generated file is whether the package is
 listed in dsa-needed.txt and dla-needed.txt.
 
 That would be two boolean fields at the source package level (default value
 of False if missing).

agreed.

Filing this as a new bug, so we can finish #761859 first and then extend it...


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: apt-build - Authentication warning overridden. - security issue?

[Holger Levsen]

Hi,

I think you probably just need to run apt-get update before apt-get 
install...

It's definitly not a security issue deserving the attention of the security 
team.


cheers,
Holger



signature.asc
Description: This is a digitally signed message part.

Re: apt-build - Authentication warning overridden. - security issue?

[Holger Levsen]

Hi,

On Donnerstag, 19. März 2015, Patrick Schleizer wrote:
  I think you probably just need to run apt-get update before apt-get
  install...
 I did that, I am sure of it. Reproduced this on two different systems.

can you put the output of apt-get update and apt-cache policy on 
paste.debian.net?


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: security-tracker json deployed

[Holger Levsen]

Hi Raphael,

On Montag, 16. März 2015, Raphael Hertzog wrote:
 I'm currently trying to use the generated json but the data below the
 releases field doesn't correspond to what we discussed. It contains
 entries like wheezy-security or squeeze-security when it was supposed
 to have only the underlying release names squeeze or wheezy.

The repository dictionary has what you are looking for. The releases 
dictionary indeed lists all versions in all existing releases. 

Maybe you would prefer the releases dict to only have keys squeeze, wheezy, 
jessie and sid and an additional sub-release key?

 Example with CVE-2014-9663 in freetype if you need one:

thanks, examples are always good to discuss these things!
 
   {
debianbug: 777656,
releases: {
 jessie: {
  status: resolved,
 },
 sid: {
  status: resolved,
 },
 squeeze-security: {
  status: open,
 },
 wheezy-security: {
  status: resolved,
 }
},
repositories: {
 jessie: 2.5.2-3,
 sid: 2.5.2-4,
 squeeze: 2.4.2-2.1+squeeze4,
 squeeze-security: 2.4.2-2.1+squeeze4,
 wheezy: 2.4.9-1.1,
 wheezy-security: 2.4.9-1.1+deb7u1
},
   },


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#762289: switching PTS links to tracker.d.o

[Holger Levsen]

Hi,

unless someone objects profoundly I'll switch the links from the security-
tracker to to tracker.debian.org instead of pointing to the old PTS in the 
coming days.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: security-tracker json deployed

[Holger Levsen]

Hi,

On Freitag, 27. Februar 2015, Paul Wise wrote:
 To clarify, I was suggesting keep the version numbers in the
 repositories section but only keep fixed version numbers in the
 releases section. Also, the fixed version numbers appear to be
 incorrect, for example the website says CVE-2012-6656 was fixed in
 eglibc 2.13-38+deb7u7 but the json says it was fixed in 2.13-38+deb7u8.
 
 https://security-tracker.debian.org/tracker/CVE-2012-6656

this is a fine example, 2.13-38+deb7u7 is indeed not returned by my queries. 
looking into this now.

(and then into json format errors...)


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Re: Bug#761859: security-tracker json deployed

[Holger Levsen]

Hi,

On Montag, 9. März 2015, Raphael Hertzog wrote:
 But I wonder why you have such problems? Aren't you storing the result
 in memory and then letting a json lib output the data?

I dont, as I've converted the previous yaml output to json, because I liked 
the humand readability of the result...
 
  Open questions:
  - should the output include description fields if the value is null?
  - should the output include nodsa fields if the value is null?
  - should the output include remote fields if the value is null?
 No to the 3 above questions.

ok, changed where applicable.
 
 That said your repositories field is weird for now... first it's an array
 and not a dictionnary for a reason that I don't understand. And the values
 contain only a dictionnary with a single key mapping codename =
 version.

it's the current version as opposed in that repo...
 
  And then I thought, urgency would be a per issue field (and thus would be
  the same for different suites), with the exception that the (suite
  specific) end- of-life information is also stored there.
  Turned out I was wrong, there are many more cases where the urgency of
  issues *is* suite-specific (plus, issues can affect several packages.)
 I looked at some of the cases you listed, but the original CVE file only
 has a single urgency... it might be that this urgency is not in line with
 the urgency retrieved from NVD but that's OK. Our urgency should override
 that one for our needs.

when there are suite specific urgencies, the json lists those...


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#761859: security-tracker json deployed

[Holger Levsen]

Hi,

On Montag, 9. März 2015, Raphael Hertzog wrote:
 I don't understand. IIRC we said the content of repositories and
 releases was supposed to have the same structure. The only difference
 was that it applied to different versions of packages.

I think the confusion might be because you stated something else than Paul..

Currently there are two dictionaries in the output, one called releases 
containing dictionaries containing information about the status of a given 
issue in a release, and another with repositories and the current version.

eg

 almanah: [
  {
   debianbug: 702905, 
   description: Almanah Diary 0.9.0 and 0.10.0 does not encrypt the 
database when closed, which allows local users to obtain sensitive information 
by reading the database., 
   issue: CVE-2013-1853, 
   releases: {
jessie: {
 status: resolved, 
 urgency: low**, 
 version: 0.9.1-1
}, 
sid: {
 status: resolved, 
 urgency: low**, 
 version: 0.9.1-1
}, 
squeeze: {
 status: resolved, 
 urgency: unimportant, 
 version: 0
}, 
wheezy: {
 status: resolved, 
 urgency: low**, 
 version: 0.9.1-1
}
   }, 
   repositories: {
jessie: 0.11.1-1, 
sid: 0.11.1-1, 
squeeze: 0.7.3-1, 
wheezy: 0.9.1-1
   }, 
   scope: local
  }
 ], 


repositories has the current versions, releases has the fixed versions if 
there are any. Oh well, why did I pick this example, sigh. so squeeze is not 
affected...

I think I will release what I have now and we can look for further needed 
tuning then.

And then I thought, urgency would be a per issue field (and thus
would be the same for different suites), with the exception that the
(suite specific) end- of-life information is also stored there.
Turned out I was wrong, there are many more cases where the urgency
of issues *is* suite-specific (plus, issues can affect several
packages.)
   
   I looked at some of the cases you listed, but the original CVE file
   only has a single urgency... it might be that this urgency is not in
   line with the urgency retrieved from NVD but that's OK. Our urgency
   should override that one for our needs.
  
  when there are suite specific urgencies, the json lists those...
 
 Well, I'm saying that I was agreeing with you. The severity ought to be a
 issue/package property, not a issue/package/repository one. And I don't
 understand the discrepancy you get because for me there are only two
 sources of urgencies:
 - those set on lines like - tcllib 1.16-dfsg-2 (low; bug #780100)
 - those coming from the NVD database

the problem is that the urgency field is abused to also hold the information 
about end-of-life, not yet assigned and unimportant, thats basically why 
urgency has to be suite specific...

(and this is why I think the db needs a redesign: it has been abused to store 
things which were not planned, and it shows.)


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: security-tracker json deployed

[Holger Levsen]

Hi,

I have deployed this now. It might be that fixed_version=0 means not 
affected but i'm not sure yet and my mind wants a break (for a moment)...


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: security-tracker json deployed

[Holger Levsen]

Hi Florian,

On Donnerstag, 26. Februar 2015, Florian Weimer wrote:
 There used to be a job that downloaded the full description from the
 NVD web service and put it into the nvd_data table (update-nvd and
 DB.updateNVD()).  The web service looks at this table and prefers the
 descriptions found there.

I've just confirmed, the job is still there and run regularily by cron.

I've also updated my local security-tracker to include the long description  
and will push this to soler shortly...


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#761859: security-tracker json deployed

[Holger Levsen]

Hi Paul,

On Donnerstag, 26. Februar 2015, Paul Wise wrote:
 I noticed the description fields are truncated, is that intentional?

that's all that is stored in the db...

 What about making the structure like this?

why? :)

 I'm guessing the code only
 produces one instance of each package.

yes

 {
   package1: {...},
   package2: {...},
 }
 
  I haven't tested the output against a json validator yet... so feedback
  welcome and I do expect some more work to do...
 
 Not a helpful error but the Python json loader tracebacks, try:
  import json
  with open('json') as f: data = json.load(f)

thanks, will give it a try later. (Currently waiting for more feedback before 
touching the code again...)

  - should the output include description fields if the value is null?
  - should the output include nodsa fields if the value is null?
  - should the output include remote fields if the value is null?
 Probably not.

ack
 
  - for the releases with issue status != resolved, should the version be
  ommitted? (as its rather meaningless then... also the repositories fields
  also contain those versions. (and those should be kept IMO)
 Hmm, I wouldn't have thought there would be a version present for
 unfixed issues? The raw data in the CVE list certainly doesn't have
 version numbers for unfixed issues.

the tracker shows them all the time, eg on 
https://security-tracker.debian.org/tracker/CVE-2013-2131

 I'm thinking omit such versions.

/me too


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: security-tracker json deployed

[Holger Levsen]

control: tags -1 + pending

Hi,

so I've deployed my patches now and you can get json at 
https://security-tracker.debian.org/tracker/data/json now.

I haven't tested the output against a json validator yet... so feedback 
welcome and I do expect some more work to do...

Important change:
- CVEs are now called issues as there are a few different issues than CVEs 
included.

Open questions:
- should the output include description fields if the value is null?
- should the output include nodsa fields if the value is null?
- should the output include remote fields if the value is null?
- for the releases with issue status != resolved, should the version be 
ommitted? (as its rather meaningless then... also the repositories fields also 
contain those versions. (and those should be kept IMO)

And then I thought, urgency would be a per issue field (and thus would be the 
same for different suites), with the exception that the (suite specific) end-
of-life information is also stored there. 

Turned out I was wrong, there are many more cases where the urgency of issues 
*is* suite-specific (plus, issues can affect several packages.)

Unedited debug output (so you can look at these issues if you really care), 
please add spaces mentally:

this should never happen unimportantlowCVE-2015-1563
this should never happen mediumhigh**CVE-2008-5234
this should never happen lowhigh**CVE-2008-5237
this should never happen mediummedium**CVE-2008-5239
this should never happen lowmedium**CVE-2008-5240
this should never happen lowmedium**CVE-2008-5241
this should never happen mediummedium**CVE-2008-5242
this should never happen medium**lowCVE-2011-2516
this should never happen high**lowCVE-2013-1436
this should never happen medium**lowCVE-2011-4613
this should never happen not yet assignedlowTEMP-000-269968
this should never happen low**lowCVE-2011-4028
this should never happen unimportanthighCVE-2012-0064
this should never happen unimportanthigh**CVE-2012-2118
this should never happen medium**lowCVE-2013-6424
this should never happen unimportantnot yet assignedCVE-2014-8103


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: security tracker json...

[Holger Levsen]

Hi Raphael,

thanks for your feedback! I got a consistent idea now.

On Mittwoch, 25. Februar 2015, Raphael Hertzog wrote:
  - if a CVE is neither fixed in lts/security/(squeeze|wheezy), but the
  version in lts/security differs from squeeze|wheezy, which version+suite
  to display as affected?
 The aggregate view should use the latest version available from all the
 repositories associated to the release of interest.

so it would display:

version: $some_version_in_lts
release: squeeze

which is kinda wrong/inaccurate...

But ok, I'll implemented both outputs (full+aggregated) and then we can see if 
this is a bug or a feature :)


cheers,
Holger





signature.asc
Description: This is a digitally signed message part.

Bug#761859: prototype ready

[Holger Levsen]

Hi,

On Dienstag, 24. Februar 2015, Paul Wise wrote:
 I think it would be useful to provide the non-aggregated version for
 folks who only use some of the stable suites. Not sure if the sectracker
 has information about stable-proposed-updates but if so it would be good
 to include it too.

it hasn't, see #645201 track uploads to proposed-updates


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#761859: prototype ready

[Holger Levsen]

Hi,

On Dienstag, 24. Februar 2015, Richard Hartmann wrote:
 Depending on your layout, you don't really need two different JSON
 files, though.

how would you distinguish between squeeze, which includes lts and security, 
and squeeze, which doesnt? Same for wheezy (and security and not).


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: prototype ready

[Holger Levsen]

Hi,

On Montag, 23. Februar 2015, Raphael Hertzog wrote:
 The only missing data I see is the Debian bug report assigned to each CVE.

I'll add that.
 
 And you call the file json but it contains YAML :-)

yeah, fixed in the last attached patch, but I will rewrite it to actually 
output json...

 Otherwise, I see that you have the raw data per real suite (aka squeeze is
 never fixed, only squeeze-lts is fixed) and I would prefer having data
 consolidated by release (i.e. you get the squeeze status by merging
 squeeze, squeeze-security and squeeze-lts, wheezy by merging wheezy and
 wheezy-security, etc.).
 
 Is that possible ?

surely. I just wasn't sure whether this should be done on the security-tracker 
side or by it's users... or I could provide two versions: json-full and json(-
aggregated) - do you think that would be useful?


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#761859: prototype ready

[Holger Levsen]

Hi,

On Montag, 23. Februar 2015, Paul Wise wrote:
 Hmm, it appears that these are the default urgency from NVD and the ones
 without asterisks are ones set by SVN committers. That doesn't appear to
 be a distinction worth preserving but it is fine to do so.

I kept it under the premise of presenting the raw data.
 
 Please ensure that this json is linked to from the front page of the
 security tracker and from the security tracker documentation so that
 people building on it can find it easily.

will do.

 I think for other consumers of the data (not distro-tracker), exposing
 fixed version numbers might be interesting. For instance, someone with
 500 machines who aggregates host/package/version information and then
 correlates that with the list of security issues from the sectracker.

i'll include this in the detailed json output.

 I should stop bike-shedding though :)

:)

 Anyway, the current JSON is good for the distro-tracker from a content
 perspective (so please deploy)

will do RSN :)


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761859: yaml...

[Holger Levsen]

Hi,

the patch currently creates yaml, not json. Which do you prefer?

Also, is the bug description useful in the data? Do you want no 
data/remote/local or (null|None)/true/false?

Anything else? 


cheers,
Holger
From 4237fa854c9dc4f1d8ac8de5c8e2030f68bf847b Mon Sep 17 00:00:00 2001
From: Holger Levsen hol...@layer-acht.org
Date: Sun, 22 Feb 2015 00:39:00 +0100
Subject: [PATCH] Dump data as .yaml via /tracker/data/yaml (Closes: #761859)

---
 bin/tracker_service.py | 48 
 1 file changed, 48 insertions(+)

diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index ec7cee5..fcc5621 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -138,6 +138,7 @@ class TrackerService(webservice_base_class):
 self.register('data/funny-versions', self.page_data_funny_versions)
 self.register('data/fake-names', self.page_data_fake_names)
 self.register('data/pts/1', self.page_data_pts)
+self.register('data/yaml', self.page_yaml)
 self.register('debsecan/**', self.page_debsecan)
 self.register('data/report', self.page_report)
 self.register('style.css', self.page_style_css)
@@ -1226,6 +1227,53 @@ Debian bug number.'''),
 data.append('\n')
 return BinaryResult(''.join(data),'application/octet-stream')
 
+def page_yaml(self, path, params, url):
+data = []
+old_pkg = ''
+releases = ('sid', 'jessie', 'wheezy', 'squeeze')
+for (pkg, bug, desc, release, subrelease, status, urgency, remote, nodsa) in self.db.cursor().execute(
+SELECT sp.name, st.bug_name, bugs.description,
+sp.release, sp.subrelease, st.vulnerable, st.urgency,
+(SELECT range_remote FROM nvd_data
+WHERE cve_name = st.bug_name),
+(SELECT comment FROM package_notes_nodsa AS nd
+WHERE nd.package = sp.name AND nd.release = sp.release
+AND nd.bug_name = st.bug_name) AS nodsa
+FROM source_package_status AS st, source_packages AS sp, bugs
+WHERE sp.rowid = st.package AND st.bug_name = bugs.name
+AND ( sp.release = ? OR sp.release = ? OR sp.release = ?
+OR sp.release = ? )
+ORDER BY sp.name, st.bug_name, sp.release, sp.subrelease , releases):
+
+if old_pkg != pkg:
+old_pkg = pkg
+old_bug = ''
+data.append(pkg+':\n')
+if old_bug != bug:
+old_bug = bug
+data.append('  '+bug+':\n')
+data.append('description: '+desc+'\n')
+data.append('releases: \n')
+if subrelease == '':
+my_release = release
+else:
+my_release = release+'-'+subrelease
+data.append('  '+my_release+':\n')
+if status  0:
+data.append('status: open\n')
+else:
+data.append('status: resolved\n')
+data.append('urgency: '+urgency+'\n')
+if str(remote) == 'None':
+data.append('range: no data\n')
+elif remote == 1:
+data.append('range: remote\n')
+else:
+data.append('range: local\n')
+if str(nodsa) != 'None':
+data.append('nodsa: '+nodsa+'\n')
+return BinaryResult(''.join(data),'application/octet-stream')
+
 def page_debsecan(self, path, params, url):
 obj = '/'.join(path)
 data = self.db.getDebsecan(obj)
-- 
1.9.1



signature.asc
Description: This is a digitally signed message part.

Re: Security EOL within Debian Stable

[Holger Levsen]

On Samstag, 7. Februar 2015, Jan Wagner wrote:
 it would be great if you would open a bug against the
 debian-security-support package if there isn't one pending yet.

#776904 please mark chromium as unsupported in wheezy


signature.asc
Description: This is a digitally signed message part.

Re: [SECURITY] [DSA 3148-1] chromium-browser end of life

[Holger Levsen]

Hi,

On Donnerstag, 5. Februar 2015, Paul van der Vlis wrote:
 There was always a year security support for oldstable.

you are right with that.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: [SECURITY] [DSA 3148-1] chromium-browser end of life

[Holger Levsen]

Hi,

On Donnerstag, 5. Februar 2015, Paul van der Vlis wrote:
 Iceweasel support for oldstable stopped at 24 Mar 2009:
 Icedove support for oldstable stopped at 12 Jul 2009:
 Icedove security support for oldstable stopped at 09 Mar 2011:
 The security support of Iceweasel for oldstable stopped at 26 Jun 2013:
 Limited security updates for Icedove for oldstable on 29 Aug 2013:
 No updates for oldstable anymore on 22 Apr 2014:

and then finally, sometime later in 2014, security support for oldstable was 
finally introduced for the first time.

I think you have (had?) wrong expectations.

 This is always a problem for me, because
 I have customers with complex desktop systems.

Maybe you should contribute to oldstable then, if that is what your customers 
are using.

Or just use one of the suggested alternatives in this thread. VMs were not 
mentioned explicitly, but thats also an option.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: Security Tracker Updates For Clamav

[Holger Levsen]

updated in r30231, thanks Scott!




signature.asc
Description: This is a digitally signed message part.

Re: Guidance on no-dsa and adding entries to dsa/dla-needed.txt

[Holger Levsen]

Hi,

On Dienstag, 23. September 2014, Michael Gilbert wrote:
 There is a page that lists candidates for DTSA (Debian Testing
 Security Announcements), which aren't actually done anymore

I can remove it, if it's really not used at all anymore.

 , but
 something like that would be very useful for DSA and DLA candidates.

how were those candidates determined?

 Then the separate text files could go away, and we can just use
 no-dsa in the CVE list to keep those pages up to date.

you mean those dsa-needed.txt and dla-needed.txt files?


cheers, 
Holger




signature.asc
Description: This is a digitally signed message part.

Re: Guidance on no-dsa and adding entries to dsa/dla-needed.txt

[Holger Levsen]

Hi Raphael,

thanks for your work on triaging oldstable related CVEs!

On Montag, 22. September 2014, Raphael Hertzog wrote:
 1/ is there a page on the security tracker that lists packages with
 open vulnerabilities in stable/oldstable which are neither unimportant,
 nor marked no-dsa and not present in dsa/dla-needed ? (I could not
 find one)

I have patches very much pending (=I will probably bring them live today or if 
not today, then tomorrow) which allow you to set proper filters for 
https://security-tracker.debian.org/tracker/status/release/oldstable so you 
can there see what you wanna see.
 

cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#742382: Display oldstable/stable security and olstable-lts repositories in tabular view. (Closes: #742382)

[Holger Levsen]

Hi,

On Montag, 22. September 2014, Christoph Biedl wrote:
 While the new appearence of the security tracker is a *huge*
 improvemnt, both in information details and design, thanks for that,

thanks!

 As a suggestion for the above issue:
 
 + squeeze, squeeze (security)   5.04-5+squeeze5 [gray]No longer supported¹
 | squeeze (lts) 5.04-5+squeeze7 [green]fixed
 + wheezy5.11-2+deb7u3   [light red]fix pending²
 | wheezy (security) 5.11-2+deb7u5   [green]fixed
 | jessie, sid   1:5.19-2[green]fixed

I like the idea of using more colors...
 
 + ¹ The squeeze suite has been discontinued. Use the squeeze-lts version

That's (slightly) misleading and wrong, though.

 + ² Will be handled in due course. Use the wheezy (security) version
 The footnotes are part of the text. And yes, they'd have to appear
 on every page.
 Your opinion on that?

yes, true, the security tracker still has some bugs which need to be fixed. 
Specific suggestions (like colors or footnotes) are best suggested in seperate 
short bugs, yet best with patches :-)

That said, I don't agree with the described urgency / panic. Debian might look 
bad because of bad things we do or good things we dont do, but seldomly 
because our security tracker is too accurate (or even inaccurate/wrong at 
times) :-) 


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#642987: EOL-support patch updated, to apply against new checkboxes code

[Holger Levsen]

Hi,

see mail subject and attached file.

[00:53]   h01ger | buxy: i have a patch to display end-of-life too, 
#642987 - i just dont like abusing urgency for it as i do. i'd rather have 
florians db remodelling..

but I might still commit this one to svn, as perfect is the enemy of good also 
here, and the EOL code can also be refactored, once the modell is redone :)


cheers,
Holger
From a96948b3ef4e4a40107cc8f00b9af584b6d26fb6 Mon Sep 17 00:00:00 2001
From: Holger Levsen hol...@layer-acht.org
Date: Sat, 13 Sep 2014 02:02:42 +0200
Subject: [PATCH] Display end-of-life information in the web view. (Closes:
 #642987)

---
 bin/tracker_service.py| 7 ++-
 lib/python/bugs.py| 4 ++--
 lib/python/security_db.py | 8 +---
 3 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index d3c8b10..83a53bd 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -29,6 +29,7 @@ class BugFilter:
('low_urgency', 'low', 'urgency'),
('unimportant_urgency', 'unimportant', 'urgency'),
('unassigned_urgency', 'not_yet_assigned', 'urgency'),
+   ('endoflife_urgency', 'end-of-life', 'urgency'),
 
('remote', 'hide remote scope', 'scope'),
('local', 'hide local scope', 'scope'),
@@ -76,7 +77,9 @@ class BugFilter:
 and urg == 'unimportant'
 filteruna = not self.params['unassigned_urgency'] \
 and urg ==  'not yet assigned'
-return filterlow or filtermed or filterhigh or filterund or filteruni or filteruna
+filterend = not self.params['endoflife_urgency'] \
+and urg == 'end-of-life'
+return filterlow or filtermed or filterhigh or filterund or filteruni or filteruna or filterend
 
 def remoteFiltered(self, remote):
 	filterr = self.params['remote'] and remote and remote is not None
@@ -420,6 +423,8 @@ data source.)],
 else:
 rel = '(unstable)'
 urgency = str(n.urgency)
+		if urgency == 'end-of-life':
+			urgency = self.make_red('end-of-life')
 if n.fixed_version:
 ver = str(n.fixed_version)
 if ver == '0':
diff --git a/lib/python/bugs.py b/lib/python/bugs.py
index a147e74..9247085 100644
--- a/lib/python/bugs.py
+++ b/lib/python/bugs.py
@@ -24,7 +24,7 @@ class Urgency(debian_support.PseudoEnum): pass
 
 def listUrgencies():
 urgencies = {}
-urgs = ('high', 'medium', 'low', 'unimportant', 'not yet assigned')
+urgs = ('high', 'medium', 'low', 'unimportant', 'end-of-life', 'not yet assigned')
 for u in range(len(urgs)):
 urgencies[urgs[u]] = Urgency(urgs[u], -u)
 Urgency.urgencies = urgencies
@@ -579,7 +579,7 @@ class FileBase(debian_support.PackageFile):
 comments.append(('NOTE', r))
 elif v == 'end-of-life':
 pkg_notes.append(PackageNoteParsed
- (p, '0', 'unimportant',
+ (p, None, 'end-of-life',
   release=release))
 if d:
 # Not exactly ideal, but we have to
diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index 088d4b5..52abb93 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -274,7 +274,7 @@ class DB:
  subrelease TEXT NOT NULL,
  status TEXT NOT NULL
  CHECK (status IN ('vulnerable', 'fixed', 'unknown', 'undetermined',
-   'partially-fixed', 'todo')),
+   'partially-fixed', 'todo', 'end-of-life')),
  reason TEXT NOT NULL,
  PRIMARY KEY (bug_name, release, subrelease)))
 
@@ -1305,7 +1305,8 @@ class DB:
 AND n.id = vulnlist.note
 ORDER BY vulnlist.package)):
 if fixed_version == '0' or urgency == 'unimportant' \
-   or kind not in ('source', 'binary', 'unknown'):
+or urgency == 'end-of-life' \
+or kind not in ('source', 'binary', 'unknown'):
 continue
 
 # Normalize FAKE-* names a bit.  The line number (which
@@ -1500,7 +1501,8 @@ class DB:
 # packages as vulnerable.  (If unstable_fixed == '0',
 # release-specific annotations cannot create
 # vulnerabilities, either.)
-if total_urgency == 'unimportant' or unstable_fixed == '0':
+if total_urgency == 'unimportant' or unstable_fixed == '0' \
+or total_urgency == 'end-of-life':
 continue

Bug#762214: security-tracker: sort Available releases view correctly

[Holger Levsen]

package: security-tracker
severity: minor

Hi,

the attached non-intrusive patch basically rewrites the availableRelease() 
function which is only used to create 
https://security-tracker.debian.org/tracker/data/releases which currently
is not ordered at all. The patch makes it logically by release, subrelease
and archive.

Shall I push this patch into SVN?


cheers,
Holger, finally finished chasing what he thought was a low hanging 
fruit ;)
From f1841ee6be909cd6c8e8c8bf94385edf9637954f Mon Sep 17 00:00:00 2001
From: Holger Levsen hol...@layer-acht.org
Date: Fri, 19 Sep 2014 17:02:36 +0200
Subject: [PATCH] rewrite DB.availableReleases() to make it possible to sort by
 release, subrelease and archive

---
 bin/tracker_service.py|  2 ++
 lib/python/security_db.py | 49 +++
 2 files changed, 34 insertions(+), 17 deletions(-)

diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index 4ad08be..4e87dc1 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -1141,6 +1141,8 @@ not unimportant.),
 sources = 'yes'
 else:
 sources = 'no'
+if 'source' in archs:
+archs.remove('source')
 yield rel, subrel, archive, sources, make_list(archs)
 return self.create_page(
 url, Available releases,
diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index 4917b46..1abfb8a 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -440,6 +440,14 @@ class DB:
 return -1
 self.db.createscalarfunction(subrelease_to_number, subrelease_to_number, 1)
 
+archives = ['main', 'contrib', 'non-free']
+def archive_to_number(u):
+try:
+return archives.index(u)
+except ValueError:
+return -1
+self.db.createscalarfunction(archive_to_number, archive_to_number, 1)
+
 def release_name(release, subrelease, archive):
 if archive  'main':
 release = release + '/' + archive
@@ -451,6 +459,10 @@ class DB:
 
 self.db.createcollation(version, debian_support.version_compare)
 
+def source_arch():
+return source
+self.db.createscalarfunction(source_arch, source_arch, 0)
+
 def filePrint(self, filename):
 Returns a fingerprint string for filename.
 
@@ -860,24 +872,27 @@ class DB:
 if cursor is None:
 cursor = self.cursor()
 
-releases = {}
-for r in cursor.execute(
-SELECT DISTINCT release, subrelease, archive
-FROM source_packages):
-releases[r] = (True, [])
-
-for (rel, subrel, archive, archs) in cursor.execute(
-SELECT DISTINCT release, subrelease, archive, archs
-FROM binary_packages):
-key = (rel, subrel, archive)
-if not releases.has_key(key):
-releases[key] = (False, [])
-releases[key][1][:] = mergeLists(releases[key][1], archs)
-
 result = []
-for ((rel, subrel, archive), (sources, archs)) in releases.items():
-result.append((rel, subrel, archive, sources, archs))
-result.sort()
+result.append(('', '', '', False, []))
+for (rel, subrel, archive, archs) in cursor.execute(
+SELECT * FROM
+(SELECT DISTINCT release, subrelease, archive, archs
+FROM binary_packages
+UNION SELECT DISTINCT release, subrelease, archive, source_arch() as archs
+FROM source_packages)
+ORDER BY release_to_number(release), subrelease_to_number(subrelease), archive_to_number(archive)):
+	if source in archs:
+	sources=True
+else:
+sources=False
+(p_rel, p_subrel, p_archive, p_sources, p_archs) = result.pop()
+if rel == p_rel and subrel == p_subrel and archive == p_archive:
+sources = sources or p_sources
+result.append((rel, subrel, archive, sources, mergeLists(p_archs, archs)))
+else:
+result.append((p_rel, p_subrel, p_archive, p_sources, mergeLists([], p_archs)))
+result.append((rel, subrel, archive, sources, mergeLists([], archs)))
+result.pop(0)
 
 return result
 
-- 
1.9.1



signature.asc
Description: This is a digitally signed message part.

Bug#664866: patch for: Include squeeze- and wheezy-backports in issue and package views. (Closes: #664866)

[Holger Levsen]

Hi Salvatore,

On Donnerstag, 18. September 2014, Salvatore Bonaccorso wrote:
 Disclaimer, only gave a quick look. Thanks again for the work :).

:-)
 
 I noticed when checking some random packages, that the version
 information tough is not correct. I take again the bind9 example for
 CVE-2014-0591.

yes, I'm aware of this.

 I guess this is not directly a problem of the patch, but more what it
 uncovers?

yes

 Without having digged into it: Is the problem that when
 backports is now considered as a subrelease, we will have the sorting
 of the versions

no, it's that the bug tables don't know about backports already... I'll work 
on a fix shortly... (I use+maintain backports myself, so I'm interested in 
correct functionality.)

 Thus for now (clearly) I'm not sure we really should include
 -backports ...

yes, though 
https://security-tracker.debian.org/tracker/status/release/stable-backports
https://security-tracker.debian.org/tracker/status/release/oldstable-backports

already exist and are broken. It would be trivial to disable/hide them, but 
I'm really more interested in fixing them.

On a related note, I've also reworked the selection logic on those 
status/release/$RELEASE views and replace those link logic with proper 
checkboxes (so one can select to only view high or low urgency bugs or 
whatever) and in future there could be checkboxes for sloopy-backports instead 
of regular ones, or the inclusion/exclusion of proposed updates.


cheers,
Holger



signature.asc
Description: This is a digitally signed message part.

security issues in backports (Re: [SECURITY] [DSA 3027-1] libav security update

[Holger Levsen]

Hi,

On Donnerstag, 18. September 2014, Henrique de Moraes Holschuh wrote:
 There is one thing that would be of great value:  We need someone to go
 over the debian-backports packages for pending security updates, and
 notify the maintainers of the backports or the backports ML.

I'm working on getting 
https://security-tracker.debian.org/tracker/status/release/stable-backports 
meaningful for this task. Give me some more days... ;-)
 
 Currently, at least file and libav are vulnerable in debian-backports.
 It is likely that other packages in debian-backports also require updates.

oh, yes! :/


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Re: security issues in backports (Re: [SECURITY] [DSA 3027-1] libav security update

[Holger Levsen]

Hi,

On Donnerstag, 18. September 2014, Holger Levsen wrote:
 I'm working on getting
 https://security-tracker.debian.org/tracker/status/release/stable-backport
 s meaningful for this task. Give me some more days... ;-)

for those not familar with the current security-tracker development: for the 
regular suites (oldstable, stable, testing and unstable) the above url works 
nicely, just for (oldstable|stable)-backports its currently not correctly 
implemented and thus broken.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#610220: the remaining small issue is not really pending

[Holger Levsen]

control: tags -1 - pending
# rather help is welcome to fix improve the regex as described in the bug log 
# (see previous mail to the bug)



signature.asc
Description: This is a digitally signed message part.

Bug#761889: decide about desired ordering of releases and issues

[Holger Levsen]

package: security-tracker

Hi,

the ordering of the releases (sid, jessie, wheezy...) and issues (open and 
resolved CVEs, DSAs, etc) is not consistent in the tracker web ui (and was 
undeterministic in parts).

So what do we have, there are basically two views:

package-centric, like https://security-tracker.debian.org/tracker/source-
package/bind9

and issue-centric, like https://security-
tracker.debian.org/tracker/CVE-2014-0591

Both list the releases in their page header, the issue-view lists oldest 
release on top, the package view is undeterministic (aka buggy, compare bind9 
vs linux). So that issue #1.

The issue-view then lists affected releases, also with oldest release on top. 
Then it lists releases with fixed versions, with the newest releases on top - 
no, actually unsorted. So thats #2

So that should probably be fixed to also list the oldest release on top. 
Agreed?

Then, the package view lists releases in the open issues table, with the 
oldest on the left.

So except for this one issue, the releases are ordered consistently now.

Second question: is that the prefered ordering, or should newer release be on 
the left/top? That's #3 even though it's just a question, thats one of the 
main questions to decide here!

The second main question is the issue ordering:

In the issue view, open issues, open unimportant issues and resolved 
issues are all sorted with the oldest on top. Security annoncements are 
sorted with the newest on top.

I think it's rather clear, that resolved issues should be sorted with oldest 
at bottom, like the announcements. Thats #4.

Debatable (but sadly so far only debated between Salvatore and me) is whether 
to list newer open (unimportant) issues on top or at the bottom. Salvatores 
argues that currently it's easier to see what old issues havent been handled, 
while my arguing is that new issues should be easier to see, as old ones are 
probably known already anyway. This is #5 for the team to decide :-)

I can fix #1+#2 to make the ordering deterministic, but the team should really 
decide on #3-5. Are there regular irc meetings where this could happen? Or 
else, how?


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: Switching the tracker to git

[Holger Levsen]

Hi,

On Montag, 15. September 2014, Thijs Kinkhorst wrote:
  What would be the actual benefits of moving to Git and I'm not talking

git log, git show, git stash and git branch and cherry-pick...!!

Working with a decentralized and fast(!) version control system locally is 
so much more fun + effective, the difference is hard to imagine if you 
haven't experienced it yourself.

 Some points of attention:

I've updated org/TODO now with the points raised by Salvatore and Thijs.

Just one thing made me suspicious:

 - Two main non-human use of svn are the joeyh commit script and the
 tracker itself.

the two main?? Are there others? Currently this part of TODO reads:

Security Tracker svn to git conversion
 - TBD: add here the todo items to be considered for the move
   * joeyh's commit script needs to be adopted to git
*  When fixing the joeyh one, I think it makes sense to move it to a role
   account on alioth (as previously discussed), rather than this personal
   account, at the same time.
   * the tracker itself needs to be adopted
   * There's also a very useful pre-commit hook that checks syntax of commits 
to data/*. This is something that also would need a place somewhere.
   * the sectracker user is subscribed to the commits mailinglists, and the 
commit messages trigger updates of the tracker.
   * http://security-team.debian.org is updated from svn, needs to be 
switched, should be simple
   * debsecan?


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: RFC: Invert ordering of issues in source package view: newest should be up

[Holger Levsen]

Hi Salvatore,

On Samstag, 13. September 2014, Salvatore Bonaccorso wrote:
 This changes the ordering in the 'Security announcements section,
 ordering it by release date of the DSA/DLA, right? So for example
 file will show with your patch:
 
 DSA / DLA  Description
 DLA-50-1   file - security update
 DSA-3021-1 file - security update
 DLA-27-1   file - security update
 [...]
 
 This looks like a good change to do, so ack at least from my side to
 do so.

Ok, I've pushed this one now to svn, so that we can focus on the less 
straightforward ones. Also, as someone said, reverting is easy, even with svn 
;)
 
 But above you mention to invert also the open and resolved CVEs by
 descending order? Why do you like to do that?

That was another commit/bug... I'll reply there.


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#610220: Show URLs in TODO/NOTE as hyperlinks in the web view

[Holger Levsen]

Hi,

On Montag, 15. September 2014, Salvatore Bonaccorso wrote:
 Hmm, would something wrapping around of the following work?

sounds like a good start...

 Considering there might be more than one matching group in each line,
 so the example holds only for a simplest case again :(

are there really examples with two urls in one line?

 cut-cut-cut-cut-cut-cut-
 import re
 string = Fixed by
 https://git.kernel.org/linus/57e68e9cd65b4b8eb4045a1e0d0746458502554c
 (v3.15-rc1) print re.search((?Purlhttps?://[^\s]+),
 string).group(url)

thats nice, now we just need the part before and after that match ;)


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#610220: Show URLs in TODO/NOTE as hyperlinks in the web view

[Holger Levsen]

control: tags -1 + pending

Hi,

see attached. This version also deals with several URLs in one note :)

It also works for all three recent examples of Salvatore.


cheers,
Holger
From 7b4ea6cc46ffc1a507d94c2a13ef3c27e3123031 Mon Sep 17 00:00:00 2001
From: Holger Levsen hol...@layer-acht.org
Date: Sat, 13 Sep 2014 00:56:17 +0200
Subject: [PATCH 1/8] Show URLs in TODO/NOTE as hyperlinks in the web view.
 (Closes: #610220)

---
 lib/python/web_support.py | 21 +++--
 1 file changed, 15 insertions(+), 6 deletions(-)

diff --git a/lib/python/web_support.py b/lib/python/web_support.py
index 72a4932..f1663a3 100644
--- a/lib/python/web_support.py
+++ b/lib/python/web_support.py
@@ -453,12 +453,21 @@ def make_table(contents, caption=None, replacement=None, introduction=None):
 
 def make_pre(lines):
 Creates a pre-formatted text area.
-r = []
-append = r.append
-for l in lines:
-append(l)
-append('\n')
-return tag('pre', ''.join(r))
+pre = []
+append = pre.append
+for line in lines:
+# turn https:// and http:// into links
+results=re.search((.*)(?Purlhttps?://[^\s]+)(.*), line)
+if results:
+for group in results.groups():
+if group.startswith('http://') or group.startswith('https://'):
+append(A(group))
+else:
+append(group)
+else:
+append(line)
+append(BR())
+return tag('pre', pre)
 
 def make_menu(convert, *entries):
 Creates an unnumbered list of hyperlinks.
-- 
1.9.1



signature.asc
Description: This is a digitally signed message part.

Bug#610220: Show URLs in TODO/NOTE as hyperlinks in the web view

[Holger Levsen]

Hi Salvatore,

On Montag, 15. September 2014, Salvatore Bonaccorso wrote:
  https://security-tracker.debian.org/tracker/CVE-2011-2825

hmpf, that works for 1 out 3, the other 2 are detected as one :/ 
 
 We only have a handfull of those, so: If you find a solution to catch
 also these then good. Otherwise we will need to workaround. Do you
 have an idea to catch also these?

no yet...
 
 Please commit this change, I will activate it on the security-tracker.

...but I will commit now and then will see if I find the cause why 
CVE-2011-2825 isnt displayed properly :)


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Bug#642987: Display end-of-live information in the web view. (Closes: #642987)

[Holger Levsen]

Hi,

updated patch attached.


cheers,
Holger


commit da14dc2780b7f3e3a1bde8cbd526eb271497fde2
Author: Holger Levsen hol...@layer-acht.org
Date:   Sat Sep 13 02:02:42 2014 +0200

Display end-of-life information in the web view. (Closes: #642987)

diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index 49c5746..da88700 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -63,10 +63,10 @@ class BugFilter:
 Returns True for urgencies that should be filtered.
 filterlow = self.params['show_medium_urgency'] and \
 urg in ('low', 'low**', 'unimportant',
-'undetermined', 'not yet assigned')
+'undetermined', 'not yet assigned', 'end-of-life')
 filtermed = self.params['show_high_urgency'] and \
 urg in ('medium', 'medium**', 'low', 'low**',
-'unimportant', 'undetermined', 'not yet assigned')
+'unimportant', 'undetermined', 'not yet assigned', 'end-of-life')
 filterund = not self.params['show_undetermined_urgency'] and vuln == 2
 filteruni = not self.params['show_unimportant_urgency'] \
 and urg == 'unimportant'
@@ -423,6 +423,8 @@ data source.)],
 else:
 rel = '(unstable)'
 urgency = str(n.urgency)
+		if urgency == 'end-of-life':
+			urgency = self.make_red('end-of-life')
 if n.fixed_version:
 ver = str(n.fixed_version)
 if ver == '0':
diff --git a/lib/python/bugs.py b/lib/python/bugs.py
index 15908dc..7258be7 100644
--- a/lib/python/bugs.py
+++ b/lib/python/bugs.py
@@ -24,7 +24,7 @@ class Urgency(debian_support.PseudoEnum): pass
 
 def listUrgencies():
 urgencies = {}
-urgs = ('high', 'medium', 'low', 'unimportant', 'not yet assigned')
+urgs = ('high', 'medium', 'low', 'unimportant', 'end-of-life', 'not yet assigned')
 for u in range(len(urgs)):
 urgencies[urgs[u]] = Urgency(urgs[u], -u)
 Urgency.urgencies = urgencies
@@ -579,7 +579,7 @@ class FileBase(debian_support.PackageFile):
 comments.append(('NOTE', r))
 elif v == 'end-of-life':
 pkg_notes.append(PackageNoteParsed
- (p, '0', 'unimportant',
+ (p, None, 'end-of-life',
   release=release))
 if d:
 # Not exactly ideal, but we have to
diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index 515f120..8b79ac6 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -273,7 +273,7 @@ class DB:
  release TEXT NOT NULL,
  status TEXT NOT NULL
  CHECK (status IN ('vulnerable', 'fixed', 'unknown', 'undetermined',
-   'partially-fixed', 'todo')),
+   'partially-fixed', 'todo', 'end-of-life')),
  reason TEXT NOT NULL,
  PRIMARY KEY (bug_name, release)))
 
@@ -1275,7 +1275,8 @@ class DB:
 AND n.id = vulnlist.note
 ORDER BY vulnlist.package)):
 if fixed_version == '0' or urgency == 'unimportant' \
-   or kind not in ('source', 'binary', 'unknown'):
+or urgency == 'end-of-life' \
+or kind not in ('source', 'binary', 'unknown'):
 continue
 
 # Normalize FAKE-* names a bit.  The line number (which
@@ -1470,7 +1471,8 @@ class DB:
 # packages as vulnerable.  (If unstable_fixed == '0',
 # release-specific annotations cannot create
 # vulnerabilities, either.)
-if total_urgency == 'unimportant' or unstable_fixed == '0':
+if total_urgency == 'unimportant' or unstable_fixed == '0' \
+or total_urgency == 'end-of-life':
 continue
 
 if unstable_fixed is None:


signature.asc
Description: This is a digitally signed message part.

Bug#664866: #664866 security-tracker: stable-backports not present in CVE and package pages

[Holger Levsen]

control: tags -1 + pending



signature.asc
Description: This is a digitally signed message part.

Bug#761730: tracker.d.o: please provide links to https://security-tracker.debian.org/tracker/source-package/$PKG

[Holger Levsen]

package: tracker.debian.org
severity: wishlist
x-debbugs-cc: debian-security-tracker@lists.debian.org

Hi,

the information gathered in the security-tracker should be displayed in the 
package tracker.d.o. 

There is an interface for it, see
https://security-tracker.debian.org/tracker/data/pts/1

This file lists source packages and the number of security issues. If there is 
none, no issues exist.

Each source package has a URL of the form 
https://security-tracker.debian.org/tracker/source-package/bind9

Please implement this linking :-)


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#664866: patch for: Include squeeze- and wheezy-backports in issue and package views. (Closes: #664866)

[Holger Levsen]

Hi,

we really need to refactor the codebase eventually ;-)

I've thought about treating backports as subrelease, but I've came to the 
conclusion that would be wrong.

See attached.


cheers,
Holger


From aaee1f290a7d96f8dcdff412fd9207b0a5a77bc2 Mon Sep 17 00:00:00 2001
From: Holger Levsen hol...@layer-acht.org
Date: Tue, 16 Sep 2014 01:08:08 +0200
Subject: [PATCH] Include squeeze- and wheezy-backports in issue and package
 views. (Closes: #664866)

---
 bin/tracker_service.py|  4 ++--
 lib/python/security_db.py | 10 +-
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index 69efa14..61552e8 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -914,7 +914,7 @@ checker to find out why they have not entered testing yet.),
 old_pkg = ''
 old_dsc = ''
 last_displayed = ''
-releases = ('sid', 'jessie', 'wheezy', 'squeeze')
+releases = ('sid', 'jessie', 'wheezy-backports', 'wheezy', 'squeeze', 'squeeze-backports')
 for (pkg_name, bug_name, release, desc) in self.db.cursor().execute(
 SELECT DISTINCT sp.name, st.bug_name, sp.release,
 bugs.description
@@ -959,7 +959,7 @@ checker to find out why they have not entered testing yet.),
 old_dsc = ''
 old_name = ''
 last_displayed = ''
-releases = ('sid', 'jessie', 'wheezy', 'squeeze')
+releases = ('sid', 'jessie', 'wheezy-backports', 'wheezy', 'squeeze', 'squeeze-backports')
 for (pkg_name, bug_name, release, desc) in self.db.cursor().execute(
 SELECT DISTINCT sp.name, st.bug_name, sp.release,
 bugs.description
diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index 8b79ac6..4130449 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -424,7 +424,7 @@ class DB:
 return 999
 self.db.createscalarfunction(urgency_to_number, urgency_to_number, 1)
 
-releases = ['potato', 'woody', 'sarge', 'etch', 'lenny', 'squeeze', 'wheezy', 'jessie', 'sid']
+releases = ['potato', 'woody', 'sarge', 'etch', 'lenny', 'squeeze', 'squeeze-backports', 'wheezy', 'wheezy-backports', 'jessie', 'sid']
 def release_to_number(u):
 try:
 return releases.index(u)
@@ -1530,7 +1530,7 @@ class DB:
 
 store_value('release/1/' + release, '\n'.join(result))
 
-for release in ('sid', 'squeeze', 'wheezy', 'jessie'):
+for release in ('sid', 'jessie', 'wheezy', 'squeeze'):
 gen_release(release)
 
 result = result_start
@@ -1580,7 +1580,7 @@ class DB:
 SELECT release_name(release, subrelease, archive)
 AS release, version FROM source_packages
 WHERE name = ?
-AND release IN ('squeeze', 'wheezy', 'jessie', 'sid')
+AND release IN ('squeeze', 'squeeze-backports', 'wheezy', 'wheezy-backports', 'jessie', 'sid')
 ORDER BY release_to_number(release), subrelease_to_number(subrelease), (pkg,)):
 yield release, version
 
@@ -1634,7 +1634,7 @@ class DB:
 p.version AS version, s.vulnerable AS vulnerable
 FROM source_package_status AS s, source_packages AS p
 WHERE s.bug_name = ? AND p.rowid = s.package
-AND release in ('squeeze', 'wheezy', 'jessie', 'sid')
+AND release in ('squeeze', 'squeeze-backports', 'wheezy', 'wheezy-backports', 'jessie', 'sid')
 ORDER BY release_to_number(p.release), p.subrelease)
 GROUP BY package, version, vulnerable
 ORDER BY package, version COLLATE version,
@@ -1684,7 +1684,7 @@ class DB:
 st.urgency = 'unimportant' OR NOT vulnerable AS unimportant
 FROM source_packages AS sp, source_package_status AS st, bugs
 WHERE sp.name = ?
-AND sp.release IN ('squeeze', 'wheezy', 'jessie', 'sid')
+AND sp.release IN ('squeeze', 'squeeze-backports', 'wheezy', 'wheezy-backports', 'jessie', 'sid')
 AND sp.subrelease  'security' AND sp.subrelease  'lts'
 AND st.package = sp.rowid
 AND bugs.name = st.bug_name
-- 
1.9.1



signature.asc
Description: This is a digitally signed message part.

Bug#611163: make generated HTML CSS-friendlier

[Holger Levsen]

control: tags -1 + pending
# *lalala*
# preview in ssh://git.debian.org/git/collab-maint/secure-testing.git 
# not yet merge ready though, but a nice preview
thanks

# mostly not my work, just very *lalala* :)


signature.asc
Description: This is a digitally signed message part.

Bug#611163: nice css: let there be patches...

[Holger Levsen]

Hi,

See attached or branch html5+external_css from  
ssh://git.debian.org/git/collab-maint/secure-testing.git

These patches turn the html into html5 and introduce a modern, slick css style 
inspired from tracker.d.o - enjoy! :)

 Feedback welcome!


cheers  thanks to Ulrike for the nice work!

Holger
From 1317d0e6a710195c3012f6b84afeebddfddfde20 Mon Sep 17 00:00:00 2001
From: Holger Levsen hol...@layer-acht.org
Date: Sun, 14 Sep 2014 22:36:54 +0200
Subject: [PATCH 1/4] tracker_service.py: add support for external css files

---
 bin/tracker_service.css   |  0
 bin/tracker_service.py| 11 +--
 lib/python/web_support.py |  6 +++---
 3 files changed, 12 insertions(+), 5 deletions(-)
 create mode 100644 bin/tracker_service.css

diff --git a/bin/tracker_service.css b/bin/tracker_service.css
new file mode 100644
index 000..e69de29
diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index bb1411a..79662b0 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -160,6 +160,13 @@ function onSearch(query) {
 self.register('data/pts/1', self.page_data_pts)
 self.register('debsecan/**', self.page_debsecan)
 self.register('data/report', self.page_report)
+self.register('style.css', self.page_style_css)
+
+def page_style_css(self, path, params, url):
+f=open('tracker_service.css', 'r')
+	content=f.read()
+	f.close()
+return BinaryResult(content,'text/css')
 
 def page_home(self, path, params, url):
 query = params.get('query', ('',))[0]
@@ -1198,13 +1205,13 @@ Debian bug number.'''),
 data.append(':')
 data.append(str(bugs))
 data.append('\n')
-return BinaryResult(''.join(data))
+return BinaryResult(''.join(data),'application/octet-stream')
 
 def page_debsecan(self, path, params, url):
 obj = '/'.join(path)
 data = self.db.getDebsecan(obj)
 if data:
-return BinaryResult(data)
+return BinaryResult(data,'application/octet-stream')
 else:
 return self.create_page(
 url, Object not found,
diff --git a/lib/python/web_support.py b/lib/python/web_support.py
index 3c3ab99..e8b055c 100644
--- a/lib/python/web_support.py
+++ b/lib/python/web_support.py
@@ -620,7 +620,7 @@ class RedirectResult(Result):
 
 class HTMLResult(Result):
 An object of this class combines a status code with HTML contents.
-def __init__(self, contents, status=200, doctype=''):
+def __init__(self, contents, doctype='', status=200):
 self.contents = contents
 self.status = status
 self.doctype = doctype
@@ -649,8 +649,8 @@ class HTMLResult(Result):
 
 class BinaryResult(Result):
 An object of this class combines a status code with HTML contents.
-def __init__(self, contents, status=200,
- mimetype='application/octet-stream'):
+def __init__(self, contents,
+ mimetype='application/octet-stream', status=200):
 self.contents = contents
 self.status = status
 self.mimetype = mimetype
-- 
1.9.1

From d172f236441c888a3e47a40363d4b1f283709a98 Mon Sep 17 00:00:00 2001
From: u451f u...@451f.org
Date: Sun, 14 Sep 2014 22:43:06 +0200
Subject: [PATCH 2/4] use modern html5 css. switch to external stylesheet.

---
 bin/tracker_service.css   | 133 ++
 bin/tracker_service.py|  55 ---
 lib/python/web_support.py |  12 -
 3 files changed, 164 insertions(+), 36 deletions(-)

diff --git a/bin/tracker_service.css b/bin/tracker_service.css
index e69de29..0e02a61 100644
--- a/bin/tracker_service.css
+++ b/bin/tracker_service.css
@@ -0,0 +1,133 @@
+html {
+	font-size: 100%;
+	-webkit-text-size-adjust: 100%;
+-ms-text-size-adjust:100%;
+}
+
+body {
+	width: 90%;
+	max-width: 1200px;
+	margin: 2em auto 1em;
+	font-family: Helvetica Neue,Helvetica,Arial,sans-serif;
+	font-size: 14px;
+	line-height: 20px;
+	color: #33;
+}
+
+header {
+	border-bottom: 1px solid crimson;
+	margin-bottom: 2em;
+}
+
+a {
+	color:#0088cc;
+	text-decoration:none;
+}
+
+a:hover, a:focus {
+	color:#005580;
+	text-decoration:underline;
+}
+
+ul, li {
+	list-style: none;
+}
+
+ul, ol {
+	padding-left: 0;
+}
+
+h1 {
+	font-size : 250%;
+	padding: 0;
+	margin: 0;
+	line-height: 1.4em;
+}
+
+h2 {
+	font-size : 110%;
+	background: crimson;
+	margin: 1em 0 0;
+	padding: 0.5em;
+	color: #fff;
+	border-top-left-radius: 0.5em;
+	border-top-right-radius: 0.5em;
+}
+
+h3 {
+	font-size : 110%;
+}
+
+table {
+	width: 100%;
+	border: 1px solid #ddd;
+	border-radius: 0.5em;
+	border-collapse: collapse;
+	box-shadow: 0 1px 3px #eee;
+	margin-bottom: 2em;
+}
+
+tr(even) {
+	background-color: #fafafa;
+}
+
+td, th {
+	text-align: left;
+	padding: 0.25em 0.5em;
+	border-bottom: 1px solid #ddd;
+	border-collapse: collapse;
+	vertical-align: top;
+}
+
+table tr:last-child td {
+	border: none;
+}
+
+th

Bug#742855: Sort releases correctly in tabular view. (Closes: #742855)

[Holger Levsen]

Hi Salvatore,

On Samstag, 13. September 2014, Salvatore Bonaccorso wrote:
 I tested the patch in my local instance. 

yeah, it's clearly the wrong patch, I attached, sorry.

 libspring-java as by now, might change in future, shows right now:
 This should be ordered (and for future releases):
 
 Bug   | wheezy | jessie | sid| Description

the instance here does so, and it also orders them within releases by '', 
'security', 'lts' :)

And that's the patch posted for #742382, which I've attached for clarity.

Regarding the patch I accidently send to this bug:

 I tested the patch in my local instance. It does sort now the CVEs in
 descending order, which was not what I meant. We had so far the oldest
 CVEs on top which this patch would changes.

I think this should still be done, newer stuff is usually more interesting (so 
here) and should thus be displayed on top. The reasoning because it has been 
like this since always is not so convincing.


cheers,
Holger

cheers,
Holger
From 808d4d51b67cf8a756c3bfbd290c2ade2d8a Mon Sep 17 00:00:00 2001
From: Holger Levsen hol...@layer-acht.org
Date: Sat, 13 Sep 2014 01:47:11 +0200
Subject: [PATCH] Display oldstable/stable security and olstable-lts
 repositories in tabular view. (Closes: #742382)

---
 bin/tracker_service.py| 13 ++---
 lib/python/security_db.py | 19 +--
 2 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index fb3fd27..48ad599 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -545,19 +545,18 @@ to improve our documentation and procedures, so feedback is welcome.)])])
 pkg = path[0]
 
 def gen_versions():
-for (releases, version) in self.db.getSourcePackageVersions(
-self.db.cursor(), pkg):
-yield ', '.join(releases), version
+for (release, version) in self.db.getSourcePackageVersions(
+self.db.cursor(), pkg):
+yield release, version
 def gen_bug_list(lst):
 for (bug, description) in lst:
 yield self.make_xref(url, bug), description
 
 suites = ()
-for (releases, version) in self.db.getSourcePackageVersions(
+for (release, version) in self.db.getSourcePackageVersions(
 self.db.cursor(), pkg):
-for r in releases:
-if r not in suites:
-suites = suites + (r,)
+if release not in suites:
+suites = suites + (release,)
 
 def gen_summary(bugs):
 for (bug, description) in bugs:
diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index 8831079..8316ef9 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -432,6 +432,14 @@ class DB:
 return -1
 self.db.createscalarfunction(release_to_number, release_to_number, 1)
 
+subreleases = ['', 'security', 'lts']
+def subrelease_to_number(u):
+try:
+return subreleases.index(u)
+except ValueError:
+return -1
+self.db.createscalarfunction(subrelease_to_number, subrelease_to_number, 1)
+
 def release_name(release, subrelease, archive):
 if archive  'main':
 release = release + '/' + archive
@@ -1566,14 +1574,13 @@ class DB:
 A generator which returns tuples (RELEASE-LIST, VERSION),
 the available versions of the source package pkg.
 
-for (releases, version) in cursor.execute(
-SELECT string_list(release) AS releases, version
-FROM (SELECT release, version FROM source_packages
+for (release, version) in cursor.execute(
+SELECT release_name(release, subrelease, archive)
+AS release, version FROM source_packages
 WHERE name = ?
 AND release IN ('squeeze', 'wheezy', 'jessie', 'sid')
-ORDER BY release_to_number(release))
-GROUP BY version, (pkg,)):
-yield releases.split(', '), version
+ORDER BY release_to_number(release), subrelease_to_number(subrelease), (pkg,)):
+yield release, version
 
 def getBinaryPackageVersions(self, cursor, pkg):
 A generator which returns tuples (RELEASE-LIST,
-- 
1.9.1



signature.asc
Description: This is a digitally signed message part.

small misc fixes

[Holger Levsen]

Hi,

attached are three small no brainer fixes I'd like to apply, please confirm :)


cheers,
Holger
Index: lib/python/bugs.py
===
--- lib/python/bugs.py	(Revision 28738)
+++ lib/python/bugs.py	(Arbeitskopie)
@@ -886,8 +886,9 @@
 return (%s-%02d-%02d % (year, month, int(day)), name, desc)
 
 def finishBug(self, bug):
+# FIXME: it seems wrong to hardcode the testing name here...
 # Convert all package notes to notes for etch (testing).
-testing = debian_support.internRelease(etch)
+testing = debian_support.internRelease(jessie)
 for n in bug.notes:
 if n.release is None:
 self.raiseSyntaxError(
Index: lib/python/security_db.py
===
--- lib/python/security_db.py	(Revision 28738)
+++ lib/python/security_db.py	(Arbeitskopie)
@@ -424,7 +424,7 @@
 return 999
 self.db.createscalarfunction(urgency_to_number, urgency_to_number, 1)
 
-releases = ['potato', 'woody', 'sarge', 'etch', 'lenny', 'squeeze', 'wheezy', 'sid']
+releases = ['potato', 'woody', 'sarge', 'etch', 'lenny', 'squeeze', 'wheezy', 'jessie', 'sid']
 def release_to_number(u):
 try:
 return releases.index(u)
Index: bin/tracker_service.py
===
--- bin/tracker_service.py	(Revision 28738)
+++ bin/tracker_service.py	(Arbeitskopie)
@@ -624,7 +624,7 @@
  H2('Security announcements'),
  make_table(gen_bug_list(self.db.getDSAsForSourcePackage
  (self.db.cursor(), pkg)),
-caption=('DSA', 'Description'),
+caption=('DSA / DLA', 'Description'),
 replacement='No known security announcements.')
  ])
 


signature.asc
Description: This is a digitally signed message part.

Re: small misc fixes

[Holger Levsen]

Hi,

On Freitag, 12. September 2014, Thijs Kinkhorst wrote:
 Looks good to me.

I've commited these now.
 
 Personally, I'd be fine with you just committing your stuff. People will
 be looking at commit messages anyway. And in case of trouble things are
 easily rolled back...

I could do that, but would like to have an ack from other people here first, 
mostly Florian. Raphael suggested me to post patches to the list first, so 
thats why I'm doign this atm...


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

RFC: Invert ordering of issues in source package view: newest should be up

[Holger Levsen]

Hi,

I think this is clearly a bugfix ;-) Please comment.

Both open and resolved issues will be inverse sorted, so that newest CVEs will 
be on top of the list.

cheers,
Holger

commit dd7b75472e00cea9759eb6554decf26c6fe8eb11
Author: Holger Levsen hol...@layer-acht.org
Date:   Sat Sep 13 01:28:00 2014 +0200

Invert ordering of issues in source package view: newest should be up.

diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index 8580d5b..b15924e 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -1690,7 +1690,8 @@ class DB:
 FROM bugs, package_notes as p
 WHERE p.bug_name = bugs.name
 AND ( bugs.name LIKE 'DSA-%' OR bugs.name LIKE 'DLA-%')
-AND p.package = ?, (package,))
+AND p.package = ?
+ORDER BY bugs.release_date DESC, (package,))
 
 
 def getTODOs(self, cursor=None, hide_check=False):




signature.asc
Description: This is a digitally signed message part.

Bug#742855: Sort releases correctly in tabular view. (Closes: #742855)

[Holger Levsen]

Hi,

commit baa7d44e460efe2b24e7b029633701cd29986d0d
Author: Holger Levsen hol...@layer-acht.org
Date:   Sat Sep 13 01:23:35 2014 +0200

Sort releases correctly in tabular view. (Closes: #742855)

diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index 9a25ad6..8580d5b 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -1682,7 +1682,7 @@ class DB:
 AND (bugs.name LIKE 'CVE-%' OR bugs.name LIKE 'TEMP-%')
 GROUP BY bugs.name, bugs.description, sp.name)
 WHERE vulnerable = ? AND unimportant = ?
-ORDER BY name, (pkg, vulnerable, unimportant))
+ORDER BY name DESC, (pkg, vulnerable, unimportant))
 
 def getDSAsForSourcePackage(self, cursor, package):
 return cursor.execute(


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#742382: Display oldstable/stable security and olstable-lts repositories in tabular view. (Closes: #742382)

[Holger Levsen]

Hi,

commit b22f1ba0cd9499e716f7b729f546a98bd4950dda
Author: Holger Levsen hol...@layer-acht.org
Date:   Sat Sep 13 01:47:11 2014 +0200

Display oldstable/stable security and olstable-lts repositories
in tabular view. (Closes: #742382)

diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index fb3fd27..48ad599 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -545,19 +545,18 @@ to improve our documentation and procedures, so feedback 
is welcome.)])])
 pkg = path[0]
 
 def gen_versions():
-for (releases, version) in self.db.getSourcePackageVersions(
-self.db.cursor(), pkg):
-yield ', '.join(releases), version
+for (release, version) in self.db.getSourcePackageVersions(
+self.db.cursor(), pkg):
+yield release, version
 def gen_bug_list(lst):
 for (bug, description) in lst:
 yield self.make_xref(url, bug), description
 
 suites = ()
-for (releases, version) in self.db.getSourcePackageVersions(
+for (release, version) in self.db.getSourcePackageVersions(
 self.db.cursor(), pkg):
-for r in releases:
-if r not in suites:
-suites = suites + (r,)
+if release not in suites:
+suites = suites + (release,)
 
 def gen_summary(bugs):
 for (bug, description) in bugs:
diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index b15924e..4a4a2b7 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -432,6 +432,14 @@ class DB:
 return -1
 self.db.createscalarfunction(release_to_number, release_to_number, 
1)
 
+subreleases = ['', 'security', 'lts']
+def subrelease_to_number(u):
+try:
+return subreleases.index(u)
+except ValueError:
+return -1
+self.db.createscalarfunction(subrelease_to_number, 
subrelease_to_number, 1)
+
 def release_name(release, subrelease, archive):
 if archive  'main':
 release = release + '/' + archive
@@ -1566,14 +1574,13 @@ class DB:
 A generator which returns tuples (RELEASE-LIST, VERSION),
 the available versions of the source package pkg.
 
-for (releases, version) in cursor.execute(
-SELECT string_list(release) AS releases, version
-FROM (SELECT release, version FROM source_packages
+for (release, version) in cursor.execute(
+SELECT release_name(release, subrelease, archive)
+AS release, version FROM source_packages
 WHERE name = ?
 AND release IN ('squeeze', 'wheezy', 'jessie', 'sid')
-ORDER BY release_to_number(release))
-GROUP BY version, (pkg,)):
-yield releases.split(', '), version
+ORDER BY release_to_number(release), 
subrelease_to_number(subrelease), (pkg,)):
+yield release, version
 
 def getBinaryPackageVersions(self, cursor, pkg):
 A generator which returns tuples (RELEASE-LIST,


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#642987: Display end-of-live information in the web view. (Closes: #642987)

[Holger Levsen]

Hi,

this patch I like the least, but it works. Probably it would be better to use 
bug status instead of urgency, not sure. I think this is an artefact of trying 
status...

+++ b/lib/python/security_db.py
  CHECK (status IN ('vulnerable', 'fixed', 'unknown', 
'undetermined',
-   'partially-fixed', 'todo')),
+   'partially-fixed', 'todo', 'end-of-life')),

I left it in for now.

commit 07399db5abecc0e5b79b70f2a0b47bb3519dabdd
Author: Holger Levsen hol...@layer-acht.org
Date:   Sat Sep 13 02:02:42 2014 +0200

Display end-of-live information in the web view. (Closes: #642987)

diff --git a/bin/tracker_service.py b/bin/tracker_service.py
index 48ad599..bb1411a 100644
--- a/bin/tracker_service.py
+++ b/bin/tracker_service.py
@@ -419,6 +419,8 @@ data source.)],
 else:
 rel = '(unstable)'
 urgency = str(n.urgency)
+   if urgency == 'end-of-life':
+   urgency = self.make_red('end-of-life')
 if n.fixed_version:
 ver = str(n.fixed_version)
 if ver == '0':
diff --git a/lib/python/bugs.py b/lib/python/bugs.py
index 15908dc..7258be7 100644
--- a/lib/python/bugs.py
+++ b/lib/python/bugs.py
@@ -24,7 +24,7 @@ class Urgency(debian_support.PseudoEnum): pass
 
 def listUrgencies():
 urgencies = {}
-urgs = ('high', 'medium', 'low', 'unimportant', 'not yet assigned')
+urgs = ('high', 'medium', 'low', 'unimportant', 'end-of-life', 'not yet 
assigned')
 for u in range(len(urgs)):
 urgencies[urgs[u]] = Urgency(urgs[u], -u)
 Urgency.urgencies = urgencies
@@ -579,7 +579,7 @@ class FileBase(debian_support.PackageFile):
 comments.append(('NOTE', r))
 elif v == 'end-of-life':
 pkg_notes.append(PackageNoteParsed
- (p, '0', 'unimportant',
+ (p, None, 'end-of-life',
   release=release))
 if d:
 # Not exactly ideal, but we have to
diff --git a/lib/python/security_db.py b/lib/python/security_db.py
index 4a4a2b7..06e3f11 100644
--- a/lib/python/security_db.py
+++ b/lib/python/security_db.py
@@ -273,7 +273,7 @@ class DB:
  release TEXT NOT NULL,
  status TEXT NOT NULL
  CHECK (status IN ('vulnerable', 'fixed', 'unknown', 
'undetermined',
-   'partially-fixed', 'todo')),
+   'partially-fixed', 'todo', 'end-of-life')),
  reason TEXT NOT NULL,
  PRIMARY KEY (bug_name, release)))
 
@@ -1275,7 +1275,8 @@ class DB:
 AND n.id = vulnlist.note
 ORDER BY vulnlist.package)):
 if fixed_version == '0' or urgency == 'unimportant' \
-   or kind not in ('source', 'binary', 'unknown'):
+or urgency == 'end-of-life' \
+or kind not in ('source', 'binary', 'unknown'):
 continue
 
 # Normalize FAKE-* names a bit.  The line number (which
@@ -1470,7 +1471,8 @@ class DB:
 # packages as vulnerable.  (If unstable_fixed == '0',
 # release-specific annotations cannot create
 # vulnerabilities, either.)
-if total_urgency == 'unimportant' or unstable_fixed == '0':
+if total_urgency == 'unimportant' or unstable_fixed == '0' \
+or total_urgency == 'end-of-life':
 continue
 
 if unstable_fixed is None:

cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761061: tracker doesnt show closed issues as done

[Holger Levsen]

Hi,

On Mittwoch, 10. September 2014, Moritz Muehlenhoff wrote:
 It's only that noone has come around to change this. But since you now
 have experience with the code base... :-)

grummel, this seems to be true ;)

from what I've said on irc just now:

 * | h01ger is happy to report that he has patched the security tracker so it 
eg shows whats fixed through lts uploads in the file package

whats funny though is, that it still doesnt know about wheezy-security
just lts :)
havent digged into the cause anymore last night, but the source_packages table 
doesn't seem hold the wheezy-security packages, yet the tracker knows which 
DSA was fixed in which version.

I'll now do some other stuff and later continue with this...

(oh, and it now just shows squeeze and squeeze-lts, as it would show wheezy 
and wheezy-security if that were in source_packages... I'm tempted to debug 
this now, but really need to do other stuff first :)


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Bug#761061: tracker doesnt show closed issues as done

[Holger Levsen]

Hi,

On Donnerstag, 11. September 2014, Holger Levsen wrote:
 (oh, and it now just shows squeeze and squeeze-lts, as it would show wheezy
 and wheezy-security if that were in source_packages... I'm tempted to debug
 this now, but really need to do other stuff first :)

grummel. and so this is fixed now too. will propose patches later for real.

(the cause for this was that I did make update-$MANY_THINGS (and even added 
a update-all target) but forgot to run make all, which is now fixed/included 
in my update-all target too. Guess those targets need some cleanup too...)


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.