Bug#1071739: marked as done (packages.debian.org: Removal of spam domain from download mirror page)

[Thomas Lange]

> On Fri, 24 May 2024 20:02:35 +0200, Holger Wansing  
> said:

> Hi Thomas,
> you fixed this in master branch.
> Are you sure about this?
> I somehow seem to remember, that debian-master branch is used for 
packages.d.o ...
you are right, debian-master seems to be the correct branch.
I will fix it also in debian-master.

Do you know if the branch master is used for anything?

-- 
 Thomas

Re: Typo in download link for Debian 32-bit DVD ISO

[Thomas Lange]

Thanks for reporting. It has been fixed, but the rebuild of the web
pages is still needed. It will happen in a few hours.


> On Thu, 23 May 2024 09:08:41 +0200, Stefano Pigozzi  said:

> This page seems to be listing an invalid download link for the 32-bit PC 
DVD-1 iso.
> This link:

> 
https://cdimage.debian.org/debian-cd/current/i386/iso-cd/debian-12.5.0-i386-DVD-1.iso

> Should instead be:

> 
https://cdimage.debian.org/debian-cd/current/i386/iso-dvd/debian-12.5.0-i386-DVD-1.iso

-- 
best regards Thomas

Bug#924139: transistion finished

[Thomas Lange]

Hi,

I've remove the old python2 scripts in english/mirror/timestamps/
The only remaining python2 script is now urlcheck.py in the cron
repository, which should not be used any more. Read TODO i nthis
directory.


Therefore I see no blockers for upgrading www-master to bookworm.

Any thoughts?

-- 
regards Thomas

Re: Broken page link for buster installation

[Thomas Lange]

> On Fri, 17 May 2024 17:14:19 +0200, Holger Wansing  
> said:

> The buster release page says "To obtain and install Debian, see the 
installation 
> information page ..." but such page is not existant.
I removed this sentence.

-- 
regards Thomas

Re: Broken page link for buster installation

[Thomas Lange]

Having LTS support does not mean that we recommend to install new
systems using the outdated release. Therefore I've removed the link.

And where is the page (which) mentioned?


> Editing the page like this is not professional (mentioning a page,
> which is not there anymore).
> Buster is still under LTS support, so not archived strictly.

-- 
regards Thomas

Re: Broken page link for buster installation

[Thomas Lange]

Yes, it's correct that this link is not working any more. You can find
the installer ISO here:
  https://cdimage.debian.org/mirror/cdimage/archive/10.13.0/amd64/iso-cd/

> On Thu, 16 May 2024 17:49:46 +, Heidi Fehr  said:

> Hello!
> I recently went to go look for an installer for Debian Buster v10 but it 
appears that the page link is no longer working.
> https://www.debian.org/releases/buster/debian-installer/

-- 
viele Grüße Thomas

Re: The problem with security newsletters and newsletters on the security center

[Thomas Lange]

> On Wed, 8 May 2024 17:44:01 +0600, Тимур Казбеков  
> said:

> Thanks.
> Is it ever possible to trust these sources, and what do you not want to 
do, and you cannot manage them?
> 
https://salsa.debian.org/security-tracker-team/security-tracker/-/raw/master/data/DSA/list
> 
https://salsa.debian.org/security-tracker-team/security-tracker/-/raw/master/data/DLA/list
These sources are the original ones, managed by the security team.
If there's some error in it, they can fix it.
You can file a bug report or send the security team an email about
errors in these files.
-- 
regards Thomas

Re: The problem with security newsletters and newsletters on the security center

[Thomas Lange]

Hi,

DSA-5248-1 is about php-twig - security update. This is correct in the
security tracker https://security-tracker.debian.org/tracker/DSA-5248-1

Unfortunally the email that was send about this security fix
https://lists.debian.org/debian-security-announce/2022/msg00217.html
had the wrong DSA number in the subject line and the mail itself.

In our web pages, we can fix this, so the security entry links to the
right email. This means www.debian.org/security/DSA-5248-1 will link
to the correct email, but this email includes a wrong DSA number.
Sorry for this confusion, but we cannot change the email afterwards.

Here's the file of rewrites/fixes we use to link to the correct emails:
https://salsa.debian.org/webmaster-team/webwml/-/blob/master/english/security/data/dsa.fixes?ref_type=heads

-- 
regards Thomas

Re: Proposal: Refresh list of category in https://lists.debian.org/

[Thomas Lange]

Hi,

splitting developers and users is a good idea in my oppinion.
In the webwml repository
g...@salsa.debian.org:webmaster-team/webwml.git
there's the directory english/MailingLists/desc which includes
description, but I'm not sure if they are used for creating the web
page lists.debian.org. The listmaster team should know better.

-- 
regards Thomas

Re: Re: Errors building /releases/buster folder in several languages (Re: webwml failed)

[Thomas Lange]

If I'm not wrong the current build errors come from these lines in 
Makefile.common:

151 ifneq "$(LANGUAGE)" "en"
152 #Ensure that MO files do exist
153 #Translators have to run make under webwml//po if they
154 #want it updated, it cannot be automatically performed.
155 $(LOCALEDESTFILES): $(LOCALEDIR)/%.mo: $(WMLBASE)/po/%.$(LANGUAGE).po
156 $(MAKE) -C $(WMLBASE)/po install-$*.mo


I wonder if this really because of a change in releases/
It looks more like the build in l10n/po fails.
-- 
regards Thomas

Re: Errors building /releases/buster folder in several languages (Re: webwml failed)

[Thomas Lange]

Thanks for helping. I already saw this error but didn't knew how to fix it.

> On Tue, 30 Apr 2024 15:26:29 +0200, Laura Arjona Reina 
>  said:

> I have logged in www-master and I am manually deleting those 
> files/folders so the /releases/ folder for the language matches the 
> structure of the git repo, and "make" only tries to build what it's 
> really needed.


-- 
regards Thomas

Re: finding unsused scripts, add comment into scripts that are still used

[Thomas Lange]

> On Fri, 19 Apr 2024 18:32:02 -0300, Matías Bellone 
>  said:

> In my experience, a better comment would be where and/or why the script 
is used so that in 2030 anyone that reads the comment can know where to start 
searching to verify if it is still
> being used.

Great idea, but please also add the current year.
-- 
best regards Thomas

finding unsused scripts, add comment into scripts that are still used

[Thomas Lange]

Hi all,

I like to find scripts in the webwml repo that are not used any more
but were not removed from the git repo.
Therefore I suggest to add a comment to those scripts, that are still
used anyone like translators or by our cron jobs.

I did this already for some script as you can see in
git show 89df02f8989c7a72150aac9ac05801f487ed226e

Please check the scripts that you are using and add the comment
"# This script is still used in 2024"

If you find scripts (in your translation directory) that you know
are not longer used, please just remove it from the git repository.

You can find all scripts by using this command:
find ! -type d ! -type l -perm -a+x


For e.g. I guess these scripts in may be removed, since the english
version was removed in 2013 and for e.g. the german version still
includes the non-existing english file.

./portuguese/News/weekly/makemail.pl
./russian/News/weekly/makemail.pl
./german/News/weekly/makemail.pl


I'm also not sure if these for e.g. are still needed. Does anyone know?
switch_to_git_translations.pl
vcs-test.pl

There are many more examples which may be obsolete.

-- 
regards Thomas

Bug#872944: #872944 www.debian.org: Remove JavaScript from Policy Manual published on web mirrors

[Thomas Lange]

Currently we have a working solution using js and providing multi page
html. That's a good solution which is already available.


> I did not go deeper into this scenario, I just found 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=877337
> which includes a forward-backword-forward dance switching multiple
> times between multi-page and single-page html variant requests.
A single page html may be an additional option but there's already the
single page txt version and the PDF. That's sufficient and I see no
need in providing more formats of this manual.

Therefore we can close this and I will close 877337.

-- 
regards Thomas

Bug#872944: #872944 www.debian.org: Remove JavaScript from Policy Manual published on web mirrors

[Thomas Lange]

> On Wed, 10 Apr 2024 21:33:50 +0200, Holger Wansing  
> said:


> The second javascript functionality is the full-text search.

> Please note, that I made use of javascript by intend, despite of this bug 
> requesting to remove all js functionality.
Hi holger,

in the past we tried to avoid javascript, but that's long time ago
(like 7 years) and nowadays I see no reason to do web pages without it
if we loose functionality.

So please go ahead and use js. I think the search function is very important.

I think it's important not to load js code on our pages from an
external URL, but to provide it from our web servers (selfhosted).

> Please note, that this decision is not only for debian-policy, but for
> all sphinx-based manuals on Debian website.
> (I hope we don't make different decisions on this question for the 
> various manuals we have. That would make the implentation once again
> more difficult.)
All sphinx-based manuals can use js from my point of view. There's no
reason to handle some manuals differently.


> What should be done now?
Close the bug that request to remove all js functionality (#876241).

-- 
regards Thomas

Bug#987943: www.debian.org: Developers Reference: Sphinx search non-functional: searchindex.js missing

[Thomas Lange]

> On Mon, 8 Apr 2024 00:35:46 +0200, Holger Wansing  
> said:

>> Can be viewed at 
>> 

>> (also with a different html theme, BTW)
>> 
The search works for me. Thanks a lot.

-- 
regards Thomas

Re: issue with Debian-style html theme for sphinx-based documents

[Thomas Lange]

Hi Holger,

even if things get more complex, this is a working solution. I'm very
happy for that and there's no need for spending more time into looking
for a perfect solution.
>From my side you get a thank you very much and a GO for applying this patch.

> On Tue, 2 Apr 2024 14:47:12 +0200, Holger Wansing  
> said:

> The 1ftpfiles and 7doc scripts, which need to be adapted for that, and
> also the situation on the www mirrors is getting more complex, so I'm 
unsure 
> if we want this.
> See my patch.

> On the other side, I don't see any other solution apart from developing
> a new theme.

-- 
regards Thomas

Re: Translation of DSA

[Thomas Lange]

Hi Giuseppe,

the DSA are untranslatable on our web page, yes. We did that on
purpose. First, these announcements are very technical information,
not intended for end users. End users just want to upgrade their
system and know if all packages are up to date. That's why we think
most people that are interested in these security information
understand english. Second, there are automatic translation services, that can
easily translate those pages.

Adding PO support may be possible (you have to change the perl
script), but I will not spend time into it, because IMO there are
more important parts on the web page I like to improve.

-- 
best regards Thomas

Bug#1067649: Verification page is not accessible from the homepage

[Thomas Lange]

> So here I adapt my original request: please add the link to /verify at
> the /distrib page; and if possible, also consider renaming the link of
I've added the link now to the /distrib web page.

-- 
regards Thomas

Bug#1067649: Verification page is not accessible from the homepage

[Thomas Lange]

I don't think we need the link on the startpage, but maybe on the
/distrib page where we provide other ISO downloads.

But in the end I try to remember whenever I have verified an ISO by
myself.

I can't remember and I guess I never did it. Maybe I
check that I'm using https://www.debian.org. That's all I need to
trust.

-- 
regards Thomas

Re: Upcoming stable point release (12.5)

[Thomas Lange]

> On Thu, 8 Feb 2024 11:25:22 +0100, Holger Wansing  
> said:

> Now leaving out the year from above link like
> https://www.debian.org/security/dsa-5612
> also redirects to the same announcement.

> @Thomas: is this the intended behaviour (both variants are supported
> and it stays this way) ?
> In that case nothing needs to be done in the point release 
announcements...

Yes, it's intended to work with and without the year. The security
team knows about this and will change to links without the year in the future.

I prefer without the year, because people do not care about the year
when looking for a DSA/DLA, and it's hard to guess the year if you
don't know it but only the number.
-- 
regards Thomas

Re: Upcoming stable point release (12.5)

[Thomas Lange]

> On Thu, 08 Feb 2024 06:32:33 +, "Adam D. Barratt" 
>  said:

> There's a map in the www.d.o Apache config, so existing URLs should
> still work afaik.

> https://www.debian.org/security/2023/dsa-5551 , for instance, from the
> 12.4 announcement at https://www.debian.org/News/2023/20231210 still
> provides a means of accessing DSA 5551.

It's now easier than before. You don't need the year in the URL any more.
This should be used instead:
https://www.debian.org/security/dsa-5551

-- 
regards Thomas

removing items from the www footer

[Thomas Lange]

Hi,

I've prepared a patch that removes some items from the footer and make
some items non bold.

I removed the items that are not that important (like Debian books, CD
vendors) or are already linked on other pages (e.g. Mailing List
Archives, Ports/Architectures).

Before applying the patch and since changing the footer will rebuilt
all web pages, I like to get some feedback.


Removed items:
< Partners
< Donations
< Data Privacy
< Contact Us
< CD vendors
< Pre-installed
< Installation manual
< Debian Books
< Mailing List Archives
< Ports/Architectures


There will be non-bold in the new version:
< Pure Blends
< Debian Packages
< Developers' Corner


Here's a screenshot of the new footer:




regards Thomas

Re: Removed pages under /releases/

[Thomas Lange]

Hi Andreas,

another reason that I forgot was that we had too many versions of the
installmanual and the release notes. For many old releases we've kept
them in 14 languages for every architecture (around 10) for each old release.
E.g.:
https://web.archive.org/web/20230625201750/https://www.debian.org/releases/jessie/releasenotes
https://web.archive.org/web/20230625201754/https://www.debian.org/releases/jessie/installmanual


That made our own search engine produce very bad results, because
often if shows results for old releases instead on hits from the
newest installation manual. Our search engine cannot sort the result
by date btw. Someone said we should fix this problem,
but as noone worked on this I tried to produce better results by
removing old content. I don't think that many people need the old
version of the installation manual of releases, that are not supported
by Debian or even by the LTS people any more. Do you think that people
need the release notes because they want to install Debian jessie
nowadays? We always have archive.org or softwareheritage for thoses
information.

In the past we've built every release note for two (or was it three?)
releases 6 times a days regardless if the sources of the release notes
had changed or not. I improved the build time a lot by adding code
that only build them (for all languages and architectures) if
something changed. Maybe you remember this improvement.

I try to have the point of view from the user of our web pages. If
they look or search for information on our web pages and get old and
outdated search results, then our web page is useless for them. The
ratio of old to new content on our webpages was very bad in the
past. That's why I try to remove old content.

-- 
regards Thomas

Re: Removed pages under /releases/

[Thomas Lange]

Hi Andreas,

I've removed those pages maily because they contain outdated obsolete
content from very old releases and people put time into maintainig
them, what I think is not worth it. I think it's sufficient to keep less
information about these releases and only in english.

I don't know if the translators missed the information.
They may have a different opinion which information about old releases
are usefull for our users.

I don't think it's worth putting time into re-adding those pages or
translating old web pages. In the past I saw multiple efforts of
translators to work old security announcements for e.g.
There are even efforts to translate voting web pages from 2010 and before.

I would like to encourage translators to do work on more important
(from my point of view) pages or topics. I think translators could do
more review of content, and help finding outdated content, outdated
manuals, broken links, obsolete listings off CD vendors, obsolete
listings of vendors of pre-installed computers,

-- 
regards Thomas


>>>>> On Tue, 23 Jan 2024 16:44:47 +0100, Andreas Rönnquist 
>>>>>  said:

> Hi -
> I have noticed that folders under the releases folder for several
> languages has been removed by Thomas Lange - and as I understand it, it
> is done on the quest to speed up the build of the webpages.

> That is fine by me, but now I see that several translators has started
> re-adding those removed files again for their languages - I thought
> that these where removed so that we should only have a english version
> of these as described above to speed up compilation.

> Is this just because these translators have missed the memo, or is it I
> who have missed some information?

new redirects for www.d.o/security and www.d.o/lts/security

[Thomas Lange]

Hi all,

we now redirect all DSA/DLA URLs under security and lts/security with
or without having the year in the path and with or without a version
to their announcement mail:
Examples:
/security/dsa-5576
/security/2023/dsa-5576-2
lts/security/2023/dla-3686-1
lts/security/dla-3686

All URLs like dsa-5576-2 or dla-3686-1 are redirected to the specified
versions of the DSA. A URL containing only a DSA/DLA number but no
version currently redirect to version -1. In the future it may
redirect to the most recent version.
All redirects are not case sensitive.


@security-tracker admins:
A page like https://security-tracker.debian.org/tracker/DSA-5576
redirects to
https://security-tracker.debian.org/tracker/DSA-5576-2
On this page you have a link to the "Source  Debian" which is a link to
https://www.debian.org/security/2023/dsa-5576.
Currently this is a wrong link to dsa-5576-1.

The easiest way would be to make the "Source Debian" links always
redirect to the announcement number including the version, but without
the year. So for
https://security-tracker.debian.org/tracker/DSA-5576-2
change this link to
https://www.debian.org/security/DSA-5576-2
similar for the DLAs and so on.

-- 
regards Thomas

Re: New script to strip out entries from languages list

[Thomas Lange]

Hi Holger,

this script looks good to me. Thanks a lot for your work on this.
I think you can merge it.

> On Mon, 1 Jan 2024 23:39:09 +0100, Holger Wansing  
> said:


> In additional steps, we could also remove entries like de_AT, de_CH, de_DE
> and only leave 'de', as mentioned by Thomas.
> But that's another discussion.
I guess we do not want to remove those languages, but merge the
variants into de. But maybe this need to be done in a different
script, maybe in gen-files.pl.

-- 
regards Thomas

Re: upcoming changes of the web pages /security and /lts/security

[Thomas Lange]

> On Wed, 27 Dec 2023 16:37:57 +0100, Hans Fredrik Nordhaug 
>  said:

> Hi! 
> This looks great, but it would be nice if the list of "Recent Advisories" 
(dsa.list) was translated - maybe using PO files?
> Currently the entries are untranslated. Example:

> [24 Dec 2023] T  DSA-5588-1 putty security update

> It would be nice if the date was translated and the common categories 
like "security update".

Since 2015 we only use "security update" as a common category for
almost all advisories. Here are some numbers
- 1700 x security update, 6 other categories for the DSAs
- 2504 x LTS security update and 12 others for the DLAs

In the end we would have a localized date, but the interesting
information of the advisory itself would still be in english.
It's not worth spending time into this effort and it would make
many things much more complicated.

I also think that automatic translation software nowadays can
translate the date including the name of month easily and even
translating the advisory shouldn't be hard.

--
best regards Thomas

Re: upcoming changes of the web pages /security and /lts/security

[Thomas Lange]

>>>>> On Mon, 25 Dec 2023 21:24:19 +, Holger Levsen  
>>>>> said:

> On Mon, Dec 25, 2023 at 09:14:51PM +0100, Thomas Lange wrote:
>> Now the security team and the LTS team do not need to manually prepare
>> a .wml and .data file for each advisory.
>> Please stop creating those files for new advisories.

> what do we need to do instead? :)
Nothing in the webwml repository.
The list of recent advisories on the web pages is now automatically
generated from the data from the security tracker.
It's important that the announcement mail is sent and that those lists
are updated:

https://salsa.debian.org/security-tracker-team/security-tracker/-/raw/master/data/DSA/list
https://salsa.debian.org/security-tracker-team/security-tracker/-/raw/master/data/DLA/list

That's all.
-- 
regards Thomas

Re: upcoming changes of the web pages /security and /lts/security

[Thomas Lange]

Hi all,

as announced on Dec 7th, I have now removed the old index.wml files
and renamed new.wml to index.wml in the webwml repository under
security/ and lts/security/.

=
IMPORTANT
=
Now the security team and the LTS team do not need to manually prepare
a .wml and .data file for each advisory.
Please stop creating those files for new advisories.
=

For the translators:
Please stop translating old advisories.
We still have to adjust the translation headers because of the
renaming from new.wml to index.wml.

A hint for the languages which did not had a translation for new.wml
until now. Here are some more infos, how I created the new.wml files:

  english/security/new.wml is a copy of english/security/index.wml with some 
changes.
  You will see the change history (including a rename from dsa.wml to new.wml) 
by
$ git log -p --follow 3160b3931961~1.. index.wml

  For lts/security/new.wml use
$ git log -p --follow a1010f1cb6fd~1.. index.wml



I still need to do some cleanup and check if everything works.
The new index.wml files are not yet created yet but this will be done
in the next hours.

-- 
best regards Thomas

Re: upcoming changes of the web pages /security and /lts/security

[Thomas Lange]

> On Fri, 8 Dec 2023 10:27:24 -0500, Roberto C. Sánchez 
>  said:

> At what point should the LTS team stop generating these? Is that
> something that we should discontinue right away, or do wait for a formal
> announcement to let us know that these should no longer be generated
> when a DLA is released?
I will announce this to both security teams after I did the change. So
please continue preparing the .wml and .data files until you'll get
the "Stop" from me.

-- 
regards Thomas

Re: how to get more recent translations for important web pages

[Thomas Lange]

> On Thu, 7 Dec 2023 20:59:08 +0100, Holger Wansing  
> said:

> [Adding debian-www to the loop]
> Hi,

>> On the other hand, it occurred to me, if it's possible, to only change 
the
>> filename from let's say index.wml to index.wml.old instead of removing 
the 
>> file (assuming that the wml build process of the website ignores such
>> files; did not check that).
>> That would make it very easy for translators, to catch up with their 
work,
>> if they find time. 
>> Of course you might say "Hey, the file is not lost, we have a git repo
>> here! No need for such trick." 
>> That's of course correct, but translators might not be as familiar with
>> such advanced usage of git as DDs are.
>> So I think it would be worse it.
The only advantage I see, is that you see that there's an old
translation, for those were we will catch up with the renaming. In the
end the translators always have to use git. Having an .outdated file they need 
to use
$ git mv xxx.outdated 

I we delete the file they could use a simple 2-line shell script we
provide for them:

#! /bin/bash

hash=$(git log -- $1|head -1|awk '{print $2 "~1" }')
git checkout $hash -- $1

Let's call it undo-delete 

> I have tested this for ../german/releases/trixie/installmanual.wml,
> renamed that into installmanual.outdated and that works, the page gets
> removed from the web, and otherwise no errors on the webwml build.
> So, maybe that would be a possible "solution", instead of removing the
> translation files completely?
We still have the problem, that we might have to rename ALL deleted
files. Otherwise the translators could never rely on seeing a
.outdated file. What if there's no .outdated? Then they have to parse through 
the git
log and check if there's nevertheless an old deleted translation.

One more point: Does thi work for a complete deleted subdirectory, as I did
some time ago with tamil, albanian,...

I'm not against your solution and I think we should give it a try and
then after a year see if this helps to get more recent
translations. Then we can made a new decision if we want continue this workflow.

-- 
viele Grüße Thomas

upcoming changes of the web pages /security and /lts/security

[Thomas Lange]

Hi all,

in the past, all security related lists (like the N recent security
advisories, crossreferences, RSS feeds, OVAL) were using the .wml and
.data files which exists for each DSA and DLA. These two files are
still created manually for each DSA and DLA.

After talking to the security team, my goal is to remove the need of
this manuall work and generate all information automatically from the
primary security sources from the Debian Security Tracker. This also
makes the security information more early available to our users
without waiting for someone to prepare the .wml and .data files.


The changes will affect the webwml repository under /security/ and
/lts/security/.


What's already done

- The new lists of DSA/DLA are currently available under
  https://www.debian.org/security/new.html#DSAS
  https://www.debian.org/lts/security/new.html#DLAS

  The two new.html pages will replace the current index.html pages
  https://www.debian.org/security/index.html  and
  https://www.debian.org/lts/security/index.html

- The new list of recent security announcements now includes a
  link to the security tracker and the original announcement
  mails. The later link is a complete new feature of the web pages.

- The RSS security feeds for DSA and DLA are not generated from the
  .wml/.data files any more but using our new script mk-dsa-dla-list.
  The RSS content now includes a link to the security tracker and to the
  announcement mail.
  Since the change on Nov 16th we did not get any complaint about this.

- The OVAL files are generated without using the .data and .wml files.
  We now parse DebianSecTracker.json and /data/DSA/list from the sec-tracker
  Thanks to Carsten for implementing this. The OVAL XML files have now
  less errors but still aren ot perfect, because they never included
  information from DLA for older releases.

- A new collection of sources of security information
  https://www.debian.org/security/new.html#infos
  including examples how to access DSA, DLA and CVE information.


TODO:

- security/new.wml and lts/security/new.wml will replace the corresponding 
index.wml
- we need more translations for these two new wml files (hints for
  translators see below)
- The crossreferences will be removed an can easily replaced by using
  data/DSA/list which is easy to parse and read.

- Create new apache redirects. Currently we have
  www.d.org/security/dsa- (only lowercase)
  to www.d.o/security//dsa-
- Currently there's no similar redirect for the DLA
- NEW redirects
  redirect www.d.org/security/dsa- to the announcement mail at
  lists.debian.org/debian-security-announce//
  A script for generating the map file already exists.
  We will do this also for the DLA.

- all security//, key-rollover/ and undated/ files will be removed
- We will keep 2020-GRUB-UEFI-SecureBoot/ and 2021-GRUB-UEFI-SecureBoot/
- No more translations of security advisories are needed
  In 2023 only we had only french translations of the DSA/DLA. No
  other language did any translation of these information in 2023.
  French indeed translated ALL DSA/DLA. Wow!
  Thanks a lot to the french translators for this great work.
- We will keep the sec announcements and translations of 2023 for
  another 6 month before deleting them.
  Older translations will be removed in a few weeks after all changes
  were made.

- The translators are asked to prepare security/new.wml
  and lts/security/new.wml for their language. We will remove the old
  index.wml for languages which do not provide translations for the
  new pages.

  Here are some more infos, how I created the new.wml files:

  english/security/new.wml is a copy of english/security/index.wml with some 
changes.
  You will see the change history (including a rename from dsa.wml to new.wml) 
by
$ git log -p --follow 3160b3931961~1.. new.wml

  For lts/security/new.wml use
$ git log -p --follow a1010f1cb6fd~1.. new.wml



A side effect of the removal of the thousands of DSA/DLA will be that
our search engine will present better results. For e.g. if you search
for "security AND tracker" most results (of the 2000) are links to DSA
and DLA, but no information about our security tracker.
another e.g. for bad search result: "firefox" the first 10 hits you get are DSA 
from 2005 to 2007
another e.g.: search for "gnome" will list a lot of old DSA for icewaesel, 
icedove and other packages

If you have any comments, feel free to contact me.

-- 
best regards Thomas

Bug#1057284: migrate get-www-stats to python3

[Thomas Lange]

Package: www.debian.org

The script get-www-stats is using python2. It does not work with
python3 because it uses counts.iteritems which must be migrated.

The script should also ignore log entries with produces 403 on the web
server. Currently this is half of them because of a lot of access to
the uncompressed oval...xml files.

We only have this python2 script in the webwml repo. Python scripts in
security/oval are already python3.

regards Thomas

Re: plan to remove translations with only few pages

[Thomas Lange]

Hi,

as announced, I now have removed the web page translations for

Esperanto (eo)
Armenian (hy)
Hebrew (he)
Lithuanian (lt)
Slovene (sl)
Tamil (ta)
Albanian (sq)

because they had very few or no translation and no recent translation activity.
I've ignored changes like http -> https and similar.

> tor. 16. nov. 2023 kl. 05:00 skrev Thomas Lange :

> [cut]
> I suggest to have a bare minimum of pages translated, for e.g. the
> startpage and some pages on level 2 of the web hierarchy, maybe the
> download page should be mandatory.

I think the active members of the i18n list should decide on their own
what rules they like to propose for a new translation of the web pages
to make it into the web repository. See my ideas above.

In the past (from 2004 to 2008) we already had some rules that removed
outdated translations. Search in the git logs for
"Automatic purge of file outdated for six months"
I like this idea, but I think this is too fast, and it should depend
on how much content was changed on this web page.

-- 
regards Thomas

Re: po statistics

[Thomas Lange]

Hi all,

thanks for all the feedback. It seems that there's more work to do
than I've expected. Currently I will not work on fixing the po
statistics, because I want to concentrate on the security pages.
So, if anyone else likes to work on this, feel free to do so.

-- 
regards Thomas

Re: po statistics

[Thomas Lange]

Hi

I'm still not sure which of the languages we need or which are just
bugs in packages. Or do we have bug in the scripts, that generat this
language list?
What about the AA_BB and AA@somestring languages?

For e.g. I wonder why we have international/l10n/po/man_DE
which links only to this po file:
https://i18n.debian.org/material/po/unstable/main/i/i2p/installer/resources/locale-man/i2p_0.9.48-1.1_man_de.po.gz

This po file cleary says
"Language: de\n"
Why is the language then called man_DE and not just "de"? Is this a
bug in our scripts?

--
Thomas

Re: po statistics

[Thomas Lange]

Hi all,

the main reason I like to clean up our web pages it to get a good
ratio between useful and useless content. If we have to much content,
it's harder to find the important content in all the web pages.

A minor reason is the build time of our web pages. I've already removed
parts of our web pages which were not needed and we could save a good
amount of build time.

I like remove the useless translations, but I still try to find out
how to match the proper combinations which does not seem easy to me.

-- 
 Thomas

po statistics

[Thomas Lange]

Hi all,

currently I'm looking at the po statistics. I think I do not understand
much of it, but I wonder why so many statistics are generated.
If I look at
/srv/www.debian.org/www/international/l10n/po
I see more than 9400 html files. Wow!
We provide the "Status of PO files for language code: de — German " in
15 different languages (ls de.*.html) and all german files (including
de_*.*.html) are 112. What is this good for?
I'll just pick one:
de_PY.nl.html is
"Toestand van de PO-bestanden voor de taalcode: de_PY — German 
@tmpl_lang@ndash; Paraguay "
who needs this?


First, why is the list of languages on page
https://www.debian.org/international/l10n/po/
so long? Shouldn't we just list the main lagnuages, without it's variants?
Should we ignore all "Unknown language"?

Let's look at these three:
https://www.debian.org/international/l10n/po/log
https://www.debian.org/international/l10n/po/log_DE
https://www.debian.org/international/l10n/po/man_DE

They list some packages for which
"translation is underway" or "already translated".
Fine.

But why do we then add a long list of package names under the section
"Packages already i18n-ed"
"These packages are translated in other languages, and then could be translated 
into your own language. "


And at the end of the web page we list again all languages, same list
as on the top of the page.


I like to know which information is really needed for the translators
then I'll try to fix the Makefile to get rid of the unneeded files.
-- 
regards Thomas

plan to remove translations with only few pages

[Thomas Lange]

Hi all,

there are some languages which only have a very few web pages
translated:

Esperanto (eo) 1
Armenian (hy)  1
Hebrew (he)3
Lithuanian (lt) 3
Slovene (sl)   0
Tamil (ta) 1


They all have one additional page translated (sitemap) but often only
a few items of it. None of them have translated our startpage
www.debian.org, only one page of all these languages is up to date.

I've checked the activity in the esperanto directory, but it's hard to
find a real translation commit. The newest I found was from 2011 for
some po files which are not used AFAICS.

The lithuanian translation of donations.wml also lists ffis.de which
does not collect money for Debian since a few years.

I suggest to have a bare minimum of pages translated, for e.g. the
startpage and some pages on level 2 of the web hierarchy, maybe the
download page should be mandatory.

What's your oppinion about removing those translations which are
just useless?

-- 
regards Thomas

Re: Security RSS icon

[Thomas Lange]

Hi all,

I've reimplemented the generation of the RSS feeds. It would be nice
if you could try to use the new feeds and send some feedback.
The URL are
https://www.debian.org/security/newdsa.rdf
https://www.debian.org/security/newdsa-long.rdf

The new feeds now include links to the original announcement
mail. This is a feature we do not provide yet in the feeds.
The security feeds will only be available in english. But we do not
need translations, because except for french we only translate the title of the 
feed
chanel but neither the title nor content of the feed items.

This change is one more step to get rid of the need for manually
creating a web page for each DSA and DLA in the future.


The URLs the the new feeds are only for a test period. If everything
works, I will rename them to the dsa.XX.rdf and dsa-long.XX.rdf names,
so the users will get the new feeds without changing anything.

The new feeds are created by the script mk-dsa-dla-list
-- 
regards Thomas

Re: Why are these web pages translated?

[Thomas Lange]

Some more information about access to the translations of the webside 
statistics.

I've checked the apache logs for /devel/website/stats
and did some statistics. All statistics are for a two weeks interval.
This is the list of the pages under /devel/website/stats with the most
hits in the apache logs (two weeks before oct, 29th). I've removed
all lines which indicates that a bot, crawler or spider accessed them. 
Therefore I used this command

egrep -v 
'Googlebot|bingbot|Bingbot|Slurp|DuckDuckBot|Baiduspider|YandexBot|ww.sogou.com|Exabo|facebot|Applebot|SeobilityBot|Scrapy|SemrushBot|/yacy.net|Spider|spider|SeznamBot|qwant.com/bot|ahrefs.com/robot|mj12bot.com|petalbot|dataforseo-bot'

 40 /devel/website/stats/fr.en.html
 46 /devel/website/stats/he
 65 /devel/website/stats/fr.en
 70 /devel/website/stats/ru.en.html
 75 /devel/website/stats/ko
 83 /devel/website/stats/el
 84 /devel/website/stats/zh-tw
 85 /devel/website/stats/cs
102 /devel/website/stats/en
110 /devel/website/stats/lt
117 /devel/website/stats/fr
131 /devel/website/stats/ro
138 /devel/website/stats/es
140 /devel/website/stats/ta
143 /devel/website/stats/nl
150 /devel/website/stats/hr
151 /devel/website/stats/de.en.html
159 /devel/website/stats/ar
161 /devel/website/stats/nl.en.html
168 /devel/website/stats/eo
188 /devel/website/stats/gl
253 /devel/website/stats/zh-cn
294 /devel/website/stats/ca
313 /devel/website/stats/pl
338 /devel/website/stats/sv
348 /devel/website/stats/el.en.html
351 /devel/website/stats/ru
358 /devel/website/stats/uk
411 /devel/website/stats/sl
445 /devel/website/stats/it
460 /devel/website/stats/da
479 /devel/website/stats/sv.en.html
591 /devel/website/stats/ja
648 /devel/website/stats/nb
934 /devel/website/stats/fi
   1075 /devel/website/stats/bg
   1733 /devel/website/stats/de
   1907 /devel/website/stats/hy
   3915 /devel/website/stats/diffstat.js
   4066 /devel/website/stats/pt
  12477 /devel/website/stats/

The first non native and non english hit in the usage statistic was
4 /devel/website/stats/lt.zh-hk.html
4 /devel/website/stats/ru.de.html

That means only 4 hits in 2 weeks! All others had less hits in the logs.

I think that this clearly shows, that most translations of these pages
are not needed. Let's just keep the native language version an the
english versions.

-- 
reagards Thomas

Re: Repeated text - in general and for security

[Thomas Lange]

>>>>> On Mon, 30 Oct 2023 21:39:20 +0100, Holger Wansing  
>>>>> said:

> Am 30. Oktober 2023 21:32:16 MEZ schrieb Thomas Lange 
:
>> Hi,
>> 
>> I've moved the duplicated text now into 
english/security/security-sources.inc
>> See git show 3121c231cfff9a41f6abfc6afdfeb2aa9435046e

> Isn't the templates directory the one, which is used for such things?
We also use several .inc files in /vote and english/security/crossreferences.wml
includes ref-table.inc which is not in the template directory.

-- 
regards Thomas

Re: Security RSS icon

[Thomas Lange]

> On Mon, 30 Oct 2023 10:22:38 +0100, Hans Fredrik Nordhaug 
>  said:

> Hi!
> What is the purpose of RSS icon, see 
https://www.debian.org/security/index.en.html Earlier it linked to 
> https://www.debian.org/security/dsa That page is not an RSS-feed. So 
maybe two things have happened:

https://www.debian.org/security/dsa is still the RSS feed if you
download it for e.g. using wget. I guess you get the web page only if
you are using a browser, which sends some language negotiation information.

The problem seems to be that we now have a dsa.en.html and dsa.en.rdf
file in the directory and our apache send different files depending on
the client. So my naming of the new web page (dsa.wml) was choosen
badly.
We can easily rename dsa.wml and ../../lts/security/dla.wml
if it causes too many problems.
-- 
regards Thomas

Re: Repeated text - in general and for security

[Thomas Lange]

Hi,

I've moved the duplicated text now into english/security/security-sources.inc
See git show 3121c231cfff9a41f6abfc6afdfeb2aa9435046e

> On Mon, 30 Oct 2023 10:25:48 +0100, Hans Fredrik Nordhaug 
>  said:

> Hi!
> english/lts/security/index.wml and english/security/dsa.wml has a block 
of repeated text as the comment in lts/security/
> index.wml clearly states.

> How do we properly fix this - here and in general? In Norwegian, I have 
created _sub_some_topic.data (with the common text)

-- 
regards Thomas

Re: Repeated text - in general and for security

[Thomas Lange]

> On Mon, 30 Oct 2023 12:08:05 +0100, Hans Fredrik Nordhaug 
>  said:

> Nice.
> I was curious if other file extensions could / should be used. ".data" is 
kind of misleading for sub parts of wml files, but using ".wml"
> doesn't work I assume - it will create new pages.
I like .inc, which is used by some wml files.

-- 
rtegards Thomas

Re: Repeated text - in general and for security

[Thomas Lange]

Hi Hans,

yes, using an include should be the proper way IMO. I guess the translations
could also use an include. I'll try to work on the english version in
the next few days.

> On Mon, 30 Oct 2023 10:25:48 +0100, Hans Fredrik Nordhaug 
>  said:

> Hi!
> english/lts/security/index.wml and english/security/dsa.wml has a block 
of repeated text as the comment in lts/security/index.wml clearly
> states.

> How do we properly fix this - here and in general? In Norwegian, I have 
created _sub_some_topic.data (with the common text) and included
> them where relevant. Is this the proper webml way? Does it break 
translation stats or other tooling?

-- 
regards Thomas

Bug#1055026: do not create web statistic for all language combinations

[Thomas Lange]

There's also the problem, that a lot of files are outdated.

After I removed some files for german manually I now see this:

https://www.debian.org/devel/website/stats/de.en.html
Last Modified: Sun, Oct 29 15:26:29 UTC 2023   Last Built: Sun, Oct 29 15:29:11 
UTC 2023

This was recreated today, but

https://www.debian.org/devel/website/stats/de.de.html
Zuletzt geändert: Dienstag den 13. Jun 2023 um 19:26:34 Uhr UTC   Zuletzt 
gebaut: Dienstag den 13. Jun 2023 um 22:59:06 Uhr UTC 

Similar for nl.nl.html and nl.de.html.


-- 
 Thomas

Bug#1055028: do not create po statistic for all language combinations

[Thomas Lange]

Package: www.debian.org

Currently we generate several hundereds of files in international/l10n/po/
A lot of them are around 2.2M big.

We have an overall of 8700 files in this directory.
Do we really need statistic like these? For e.g.
/international/l10n/po/de@hebrew
/international/l10n/po/sa.de.html

We should review if we really need all these files in there.

The directory currently needs 2.2 GB disk space.
-- 
regards Thomas

Bug#1055026: do not create web statistic for all language combinations

[Thomas Lange]

Package: www.debian.org


Currently we generate nearly 700 html files in /devel/website/stats
most of them not needed and each around 2.5M big so we have 1.6GB data
there.

The web logs show that only the file in the native language and the
engish version is accessed. There are also no link to the other translations.

For example combinations like de.fr.html, fr.de.html are not needed.
I wonder why these are generated and I guess we could save a lot of
building time in 4webwml.

-- 
regards Thomas

Bug#886792: wrong results

[Thomas Lange]

When search for

You have searched for packages that names contain linux-image in suite(s) 
stable, section(s) main, and all architectures.

https://packages.debian.org/search?keywords=linux-image=names=1=stable=main

I get
Your keyword was too generic.
Please consider using a longer keyword or more keywords.

When I click on " results have not been displayed due to the search..."

I also see packages from oldoldstable.


So it's not possible to just get the list of package names from stable

-- 
regards Thomas

Bug#442083: still valid?

[Thomas Lange]

I wonder if this bug is still valid?

-- 
 Thomas

Bug#977160: new info?

[Thomas Lange]

> I'll think a bit more about what's the right thing to do now...
Hi Joost,

is there an update on that topic?
-- 
regards Thomas

Bug#354432: what should be done now

[Thomas Lange]

Can anyone decide what to do with this bug?
Wontfix, closing or is the patch working?

-- 
regards Thomas

Bug#754601: still a problem?

[Thomas Lange]

Hi victory,

is this still a problem?

-- 
regards Thomas

Bug#772822: link to mail possible

[Thomas Lange]

There's now a new list which includes a direct link to the DSA
announcement mail and the security tracker.
This can also be done for DLAs.

https://www.debian.org/security/dsa.html#DSAS

-- 
regards Thomas

Re: new global .transignore file for webwml repo

[Thomas Lange]

I wonder what the outcome of the discussion now is? Do you want to use
the global .transignore file?

About template/debian/. This directory does not contain any po files,
it only contains wml file including perl code and define-tag.

-- 
regards Thomas

New script for generating the RSS feeds for security announcements DSA

[Thomas Lange]

Hi all,

I've added a new feature to the script english/security/mk-dsa-dla-list
which can now create the rdf files from the DSA list provided
by the security team on salsa.debian.org.
It does not need to parse the .wml and .data files in our webwml repository.

This is a next step to get rid of manual adding the security
announcements to the web page.

Currently the new feeds are temporarily available at

https://www.debian.org/security/newdsa-long.rdf
https://www.debian.org/security/newdsa.rdf

Please test if they work for you. If everything is OK, I like to
replace the current versions of the feeds with the new ones. The URLs
will remain stable and are these:

https://www.debian.org/security/dsa
https://www.debian.org/security/dsa-long


The same is possible for the DLAs, which is not yet enabled.

-- 
regards Thomas

Re: oldstable download page does not provide latest 11.8 url

[Thomas Lange]

> On Tue, 17 Oct 2023 14:55:41 +0100, Steve McIntyre  
> said:

> On Mon, Oct 16, 2023 at 12:52:13PM +0200, rema...@gmx.de wrote:
>> Hello,
>> 
>> the links on this download page are not up-to-date:
>> https://www.debian.org/releases/oldstable/debian-installer/
>> 
>> It should link to ISO version 11.8

> AFAICS here the links are all pointing at 11.8?
There were some 11.7.0 tags defined in english/releases/bullseye/release.data
This was fixed in 03c7fbc9a2efd7c9ef43919d1f68e39a6bcdd83c

-- 
regards Thomas

Bug#861781: update

[Thomas Lange]

I've removed the people not involed any more and the OVAL
organisation.

We may remove all organisations, because I'm not sure if
Debian is involved in any organisations any more.

-- 
 Thomas

Re: debian.org has broken Download link.

[Thomas Lange]

> On Sun, 8 Oct 2023 21:48:53 +0700, Dmitry  said:

> Hi!
> At the main page https://www.debian.org/ the Download link with Debian 
> logo at the right part of the page is broken.
> 
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.1.0-amd64-netinst.iso

Please reload or clean your browser cache. It should be version 12.2
now that we've had point release 12.2 yesterday.

-- 
regards Thomas

Re: webwml failed

[Thomas Lange]

Hi,

the problem is, that this make target (make -C data dsa.data) should
only be called for english. How can we do this?

> On Thu, 05 Oct 2023 20:39:32 +, Debian Webmaster 
>  said:

> /srv/www.debian.org/cron/log/wml_run.log-make -C data dsa.data
> /srv/www.debian.org/cron/log/wml_run.log-make[3]: Entering directory 
'/srv/www.debian.org/webwml/spanish/security'
> /srv/www.debian.org/cron/log/wml_run.log-make[3]: *** data: No such file 
or directory.  Stop.
> /srv/www.debian.org/cron/log/wml_run.log-make[3]: Leaving directory 
'/srv/www.debian.org/webwml/spanish/security'
> /srv/www.debian.org/cron/log/wml_run.log:make[2]: *** 
[/srv/www.debian.org/webwml/english/security/Makefile:126: mkdata] Error 2
> /srv/www.debian.org/cron/log/wml_run.log-make[2]: Leaving directory 
'/srv/www.debian.org/webwml/spanish/security'
> /srv/www.debian.org/cron/log/wml_run.log:make[1]: [../Makefile.common:84: 
security-install] Error 2 (ignored)

-- 
regards Thomas

Bug#1053508: sitemap.wml also lists external web pages, review all content

[Thomas Lange]

Package: www.debian.org

The sitemap also includes external links like qa.debian.org,
lists.d.o, even wiki.d.o But a sitemap should only list internal pages.

Also the selection seems to be very random. Why should archives of
mailing lists listed here? Is devel/developers.loc that important to
be listed on a sitemap?

Why are these listed?
- Procedures for submitting a General Resolution proposal or amendment
- Standard Resolution Procedure
It's sufficient to list a link to our vote page


Please review all content. Less is more!
-- 
regards Thomas

Bug#548024: more info, still an issue?

[Thomas Lange]

I want to know if this still is an issue (bug was opened 10+ years
ago), or did we missed to close the bug. I want to avoid that we still
need to ignore it for all the next releases.

-- 
regards Thomas

Bug#983079: blends bug

[Thomas Lange]

Hi Andreas,

can you take care about this bug?
-- 
viele Grüße Thomas

Bug#1053445: Merge request regarding 'Please migrate Release Notes to reStructuredText'

[Thomas Lange]

Hi Holger,

I really like the idea no to produce release notes for each
architecture but only one. Moving to sphinx is also nice.

Sorry, if I broke your MR, by adding code that checks if something
changed in the git repo. I think I can easily add this to your code
later. So maybe we copy your version of 7release-notes and after that
I add my code.

Do you know how long the build process takes using sphinx? I've added
the code, because the build took around 90 minutes using docbook.

Any other things I should keep an eye on?
-- 
regards Thomas

how to speedup the build process of our web pages

[Thomas Lange]

Hi,

I've patched cron/parts/7release-notes and added some code, that
checks if there were some changes to the release-notes repository.
If not, don't rebuild the release-notes. Details:
git show 4560761a19d830e..d7864a901a552a18d

In the past this part took nearly 90 minutes on www-master, six times
a day. But the last change was mid of July. If we expect no changes
until end of this year, we will save 800 hours of build time.

Next target could be cron/parts/4webwml. Does anyone knows if there is
a reason why we do not use a parallel make, using all 4 cores of
www-master?

-- 
regards Thomas

Bug#905440: add info

[Thomas Lange]

The tool is already installed in www-master.
We can use

$ git restore-mtime

A test of a git restore-mtime run on www-master took one minute.
We still have to add this to the cron jobs.

--
regards Thomas

Bug#720989: closing

[Thomas Lange]

Hi,

I think in the meantime we improved the situation a lot.
On our startpage we now have a direct link to an ISO (we do no call
it CD image any more) and we have a link to other images
(this is https://www.debian.org/distrib/) which is much cleaner
than before. It would be nice to update the german translations
https://www.debian.org/distrib/index.de.html
but then this bug should be closed.

regards Thomas

Bug#680109: did we solve a problem?

[Thomas Lange]

Hi,

I wonder if there's really a big problem we try to solve here?
Do we have more complaints since 2012?
We had a longer discussion here, but still no solution yet?

If there are not that many complaints, we should just close this bug.

regards Thomas

Bug#863579: how to proceed

[Thomas Lange]

What can we do now to solve this bug?

Is there an active wiki team?
If someone needs the contact infos, using a search engine with "debian
wiki contact" will directly find the correct page. So IMO no need to
add them to the into/organisation page.

Any comments?

Bug#859123: new DAS list

[Thomas Lange]

We now have an automatically generated list of DSA, which does not need
any manual adding the infos. See
https://www.debian.org/security/dsa.html#DSAS

This list also links to the security tracker and the security
announcement mail.

My plan is to remove the need of manual editing wml and data files for
every DSA/DLA.

regards Thomas

Re: new global .transignore file for webwml repo

[Thomas Lange]

> On Mon, 02 Oct 2023 16:00:00 +0200, Laura Arjona Reina 
>  said:

>> 
>> # here you can define global Perl regex to match file that are excluded
>> News/\d{4}

> I would keep the current year and last year at least. Not sure how we 
should maintain the .transignore file, probably a quick review each January to 
update? (TODO: document it (new year checklist)

I guess this would skip all but the last two years:

News/19
News/20[01]
News/202[01]

Yes, we should update this regex each January.

-- 
regards Thomas

Re: Files deleted under /releases/

[Thomas Lange]

> On Sun, 1 Oct 2023 18:32:53 -0300, Paulo Henrique de Lima Santana 
>  said:

> Thomas, I saw you rewrote the pages under english/releases/{etch, hamm, 
> jessie, lenny, potato, sarge, slink, squeeze, stretch, wheezy, wood}, 
> but why did you deleted all files under other languages dirs too?
Hi Paulo,
I've cleaned up the english files for these outdated releases, which
was some manual work. I don't want to do all this work for all
translations, and I think that the work to fix the translations for
such old releases is not worth the time someone has to spend for doing this.
That's why I've removed the translations.
I really like to spend the time into more quality of our web pages,
and I hope that other people like translators would spend their time
in web content that are worth it.

Next time I will write an email to debian-ww before doing such big change.

-- 
regards Thomas

Why are these web pages translated?

[Thomas Lange]

Hi,

I wonder why a page like
https://www.debian.org/international/l10n/po/de is translated to other
languages than german.  IMO an english and german version would be
sufficient. I'm sure the translations are generated by a script and it
would make sense to stop the generation of these pages if the target
language does not match the language for which the statistics is
shown.
Does anyone see any use for a combination like german po file statistics
in dansk, Nederlands, svenska, suomi,... ?

BTW, currently we have more than 9000 html files in this directory on
our web servers.

-- 
regards Thomas

new global .transignore file for webwml repo

[Thomas Lange]

Hi to all, esp. the translators,

nearly two weeks ago I've added a new feature to the global
.transignore file. We can now use Perl regex for defining which files
or directories should be ignored when creating the translation
statistics. See git show 145afba3acc8c324b54fde8942f287d28a403ab1

I like to install the new .transignore file below, so the number of pages
to translate would go down from about 12200 to around 400-500.
The statistics are listed here
https://www.debian.org/devel/website/stats/

I've already asked on the #debian-www channel for comments but I got
only the question if the regex are anchored. If I get no objection, I
like to install this .transignore file next week.


# here you can define global Perl regex to match file that are excluded
News/\d{4}
News/weekly/\d{4}
News/press/\d{4}
# exclude all consultants except for index.html
consultants/(?!index.wml)
users/(?!index.wml)
devel/constitution\.\d
devel/dmup.1.1.1.wml
vote/199
vote/20[01]
vote/\d+/suppl_
partners/(?!2023)
template/debian/
lts/security/20
crossreferences.wml
security/199
security/20
security/undated
security/key-rollover
security/pam-auth.wml
international/\w+/index.wml
international/(?!index.wml)
sitemap.wml
MailingLists/subscribe.wml
MailingLists/unsubscribe.wml



-- 
regards Thomas

RE: [EXTERNAL]Re: CVE Cross references

[Thomas Lange]

Hi,

the advantage of using the DSA list is that it's very eay to parse and
may be more up-to-date than the web page containing the
crossreferences.


> On Thu, 21 Sep 2023 13:34:33 +, "Mouer, Steve" 
>  said:

> Thank you Thomas.
> It appears that the page was fixed overnight. We will look to use the 
sources in the link you have provided for this information in the
> future

> Appreciate the help.


regards Thomas

Re: CVE Cross references

[Thomas Lange]

> On Thu, 21 Sep 2023 00:25:55 +0530, Paulo Henrique de Lima Santana 
>  said:

> Hi Steve,
> I'm sending to debian-www mailing list.

> Thomas, I think you're working on that.
I'll will try to fix that, but please use the sources listed
at https://www.debian.org/security/dsa.html#infos
in the future for getting all the CVE, DSA and DLA information.

-- 
regards Thomas

Re: The Oval public page is not accessible

[Thomas Lange]

> On Thu, 31 Aug 2023 12:22:45 +0900, "sakuma, yuki" 
>  said:

> Dear Community Members
> I wanted to reach out and kindly let you know that I seem to be 
experiencing some difficulties accessing the webpage at the following
> URL: https://www.debian.org/security/oval/
I also have the same problem. I'm a Debian Developer and will contact
some people to fix this.

-- 
regards Thomas

Re: Link to www.debian.org/doc is hard to find

[Thomas Lange]

> On Sat, 15 Jul 2023 16:17:09 +0200, Holger Wansing  
> said:

> https://www.debian.org/support.en.html is named 
> "Getting help and documentation" !!!
> So people are pointed to the support page, when they look for 
documentation.
> And then they find - nothing (about docs)!
As already pointed out, the link to the documentation is in the grey
box on the /support page which should get more attention, because it's
not only a link inside text but inside a graphical element.

We also have a link to /doc when you click any of the "More" menu on
the front page. And if you search for "debian documentation" in a
search engine, you will be directly pointed to the right link.

-- 
regards Thomas

Bug#1037488: www.debian.org: drop static/packages-site.css entirely?

[Thomas Lange]

> On Tue, 13 Jun 2023 14:27:34 +0200, Cyril Brulebois  
> said:

> Probably best to finally drop it, to avoid wasting more brain time on
> suite updates (for that particular file)?
Yes, please drop it.

-- 
viele Grüße Thomas

Re: Bug#1032453: non-free-firmware changes and documentation needs to happen after SC change

[Thomas Lange]

I totally agree that there's no need to modify the Why Debian page.

-- 
regards Thomas

Bug#1029747: Empty "Categorize using"

[Thomas Lange]

> On Fri, 27 Jan 2023 09:18:20 +0100, Holger Wansing  
> said:

> This has been there for 14 years now. Wow!
> It was added in
> 
https://salsa.debian.org/webmaster-team/webwml/-/commit/4d417d00ddf442a347b1262e718f32a3a08d8e5d

> So I assume we can just remove it.
Yep, go ahead and remove it.

-- 
regards Thomas

Bug#971623: link to wnpp.d.net

[Thomas Lange]

>>>>> On Sat, 17 Sep 2022 10:00:29 +0800, Paul Wise  said:

> On Fri, 2022-09-16 at 10:56 +0200, Thomas Lange wrote:
>> I find the idea of linking to wnpp.d.net also very valuable.

> The link is already there, alongside the link to wnpp-by-tags:

The bug report is not about adding a missing link, but about replacing
our listings with a link to the wnpp.d.net page. I support this idea.

-- 
regards Thomas

Bug#971623: link to wnpp.d.net

[Thomas Lange]

Hi,

I just looked at the wnpp.d.net page and found it very nice. Looking
at the git logs of the repo for this domain, you can see that it's
very well maintained and in 2021 the old php code was replace with
django and Python 3. So, this is better maintained than some very old
code in our webwml repository.

I find the idea of linking to wnpp.d.net also very valuable.
And maybe wnpp.d.net will become a debian.org domain in the future.

-- 
regards Thomas

Bug#1019872: patch

[Thomas Lange]

Here's a patch that removes the Diffstat column. I could not test it
because I think it will not run in my local directory.

What do you think about this patch? Does it look OK?


diff --git a/stattrans.pl b/stattrans.pl
index c02fb8a5b75..c9db9fda803 100755
--- a/stattrans.pl
+++ b/stattrans.pl
@@ -489,10 +489,8 @@ foreach $lang (@search_in) {
 } else {
 $o_body .= sprintf "%s", $hits, $base, $base;
 }
-   my $stattd = sprintf 'Click to fetch diffstat data\' 
onClick="setDiffstat(\'%s\', \'%s\', \'%s\', this)">+/-', $file, 
$transversion{"$lang/$file"}, $version{"$orig/$file"};
  if (!defined $status_db{$lang}) {
 $o_body .= sprintf "%s", $msg;
-$o_body .= $stattd;
  }
if ($msg eq 'Wrong translation 
version' || $msg eq 'The original no longer 
exists') {
  if (defined $status_db{$lang}) {
@@ -637,7 +635,6 @@ foreach $lang (@search_in) {
print HTML 'Diff';
  } else {
print HTML 'Comment'."\n";
-   print HTML 'Diffstat'."\n";
print HTML 'Git command 
line';
  }
 print HTML 'Log';

-- 
regards Thomas

Bug#1019872: remove unused scripts

[Thomas Lange]

For the record:

The function setDiffstat in diffstat.js cannot work, since it uses a
non existing domain. Therefore the code in stattrans.pl using
.onClick="setDiffstat
can be removed.

On our webserver I cannot find any occurence of
onClick="setDiffstat..." in any html file.
It seems that the code (see above) in stattrans.pl is not used

-- 
regards Thomas

Bug#1019872: remove unused scripts

[Thomas Lange]

Package: www.debian.org
Severity: normal


On IRC we had a discussion if the script
english/devel/website/stats/anoncvs-cors is still used. It seems that
more parts concerning the translations stats are not needed any more.



< Mrfai> Is this script used anywere:
english/devel/website/stats/anoncvs-cors ? I cannot find anything and would 
like to remove it
< Mrfai> There's only a link to 
http://webwml.alioth.debian.org/cgi-bin/anoncvs-cors in diffstat.js but I think 
this function is not used any more.
< pabs> `git grep` says stattrans.pl emits calls to setDiffstat in the onClick 
attribute of a  tag
< pabs> Mrfai: I think the onclick=setDiffstat can probably just be replaced 
with  or maybe the compare URLs 
< pabs> hmm, it wants diffstats not full patches...
< pabs> if the exact functionality is still wanted, then maybe the script could 
migrate to cgi.debian.org
< pabs> probably best to ask debian-i18n if the diffstat is useful
< Mrfai> pabs: But I cannot find any reference of this onlick in the html files.
< Mrfai> I did this:
< Mrfai> ...@wolkenstein:/srv/www.debian.org/www/international$ grep -r 
setDiffstat .
< Mrfai> And it found nothing
< pabs> I'm looking at stattrans again
< pabs> hmm, the Diffstat column is definitely present 
https://www.debian.org/devel/website/stats/en
< pabs> seems the diffstat link is only shown if "!defined $status_db{$lang}"
< pabs> which looks like the english/international/l10n/data/status.* files
< pabs> seems english/international/l10n/scripts/gen-files.pl creates those 
files
< pabs> those files do exist on wolkenstein
< pabs> they are symlinks to /srv/www.debian.org/cron/datafiles/status.*
< pabs> Sledge's a3fcc21a7c098056a557a095178aee9f752d6059 deleted a previous 
use of diffstat stuff, adding git command-lines instead, so it sounds like the 
web team 
wants translators to manually do diffstats 
themselves, so I guess delete the diffstat column and all the supporting 
infrastructure

Bug#934608: license of the book

[Thomas Lange]

The book is available in french and english also for bullseye
https://lescahiersdudebutant.arpinux.org/bullseye-en/

This is the license:


the beginner's handbook is published under Free License WTFPLv2
---

WTFPL

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE


https://lescahiersdudebutant.arpinux.org/buster-en/LICENCE

Re: Culling obsolete pages on wiki.debian.org

[Thomas Lange]

Hi Borden,

> On Fri, 19 Aug 2022 21:12:06 +0200 (CEST), Borden  
> said:

>  A lot of it should be removed since it will just frustrate new users and 
lead to more traffic on -users as people ask why a squeeze (or earlier) 
tutorial doesn't work on their system.

I'm also a friend of removing outdated stuff, especially if the content
was not maintained since a long time (and changing
http->https is not maintaining the content).

I tried to to some cleanup in the wiki in the past, but I stopped it,
because sometimes the discussions about keeping the article or not are
frustrating.

For e.g. I found this very strange page containing no special Debian
information but only very general info. I wanted to remove it, then
someone started to rewrite it and it took some months and a few mails
before I could delete it.
https://wiki.debian.org/WebDevelopment?action=info

I would like to see more quality in the wiki articles. I don't think
it's worth having articles about non Debian specific topics,
because often you will find this infomration somewhere else in a
much better quality. A lot of Debian developers look into the
archlinux wiki for reasons.

Do we need a page like this? https://wiki.debian.org/NFS
If I want to know things about NFS (and I use NFS a lot), how to
secure it or how to debug it, the Debian wiki pages are not a good
source of information, they would never have helped me.
It links to some very old NFS documentation, link to other outdated
wiki.debian.org pages about NFS, talk about the user space kernel
server (which does not exists any more)
This may be a good example for just remove most of the NFS infos in
the Debian wiki, they are to general, outdated, and only cover some
aspects of the topic. Other web pages already do this much better, and
we do provide Debian specific NFS information in our wiki.


I often hear we should keep old things for historical reasons.
No. Very outdated wiki or web pages are not useful and they make
our search results (on the wiki and on the web pages) to be very bad.
I prefer removing old content but if we like to keep some of it, we
should keep it on a special web domain, maybe historical.d.org.
We should aks ourself how many people are interested in reading those
old information? It it worth keeping them?
Just a few examples
https://wiki.debian.org/DebianInstaller/PowerPC/OldWorld
https://wiki.debian.org/DebConf6Planning
https://wiki.debian.org/DebConf6Soccer
https://wiki.debian.org/Format
https://wiki.debian.org/DebianInstaller/GUIToDo
https://wiki.debian.org/DebianEvents/de/2009/LinuxTag
https://wiki.debian.org/LinuxTag/DebianWomen

Are these article worth keeping them? I'm sure we have hundreds or
thousands of such articles.

Sorry, if I ranted too much about old content. I try improve the
quality of the web pages since a few years, but failed to work on the
wiki because of too much content.


I wish you luck doing some cleanup or updating articles.
-- 
best regards Thomas

Bug#658227: better search results

[Thomas Lange]

Hi Olly,

in #1015198 I also reported useless search results
similar to #658227 (still open since 2012).

I found this in the xapian docs. Do you think this would be the best
solution to get the results sorted by date? I'm not sure if it would
be easy to index all our html documents by date, since the time stamps
on the files do not reflect the date of the last modification of the
content. Do you know of any other solutions?



Sorting by Other Properties

If you want to offer a "sort by date" feature, and can arrange for
documents to be indexed in date order (or a close-enough
approximation), then you can implement a very efficient "sort by date"
feature by using a boolean search (i.e. call
enquire.set_weighting_scheme(Xapian::BoolWeight());) with
enquire.set_docid_order(Xapian::Enquire::DESCENDING); (for newest
first) or enquire.set_docid_order(Xapian::Enquire::ASCENDING); (for
oldest first). There's no inherent reason why this technique can't be
used  for sorting by something other than date, but it's usually much
easier to arrange for new documents to arrive in date order than in
other orders.

-- 
best regards Thomas

Bug#905440: created ticket for adding package at rt.d.o

[Thomas Lange]

I've ask the DSA team to add the package needed on rt.d.o. This rt id
is 8892
-- 
regards Thomas

Bug#1015198: sort search results by date

[Thomas Lange]

Package: www.debian.org

The search results from www.debian.org are sorted by relevance.
When I search for "debconf" I get results from debconf from 2010, 2012
but no results for the newest article.

The results should be sorted by relevance and date or maybe add a
checkbox if people want to see the newest results first.

-- 
regards Thomas

Re: Text on support page

[Thomas Lange]

> On Sun, 12 Jun 2022 15:23:11 -0300, Paulo Henrique de Lima Santana 
>  said:

> I'm updating the translation the page "support" and It seems on this 
topic:
> https://www.debian.org/support.en.html#maintainers
> "Documentation" and "Consultants" wew forgotten there.
> Am I right?

No, the infos about documentation and consultants were moved into the
box on the right side, but it was forgotten to remove the content in
the page itself. I just fixed that in commit
40fef20783c16a860bd0f81fc28f1f7588875cbc

-- 
regards Thomas

Re: updating the /devel/website/todo page?

[Thomas Lange]

> On Mon, 7 Mar 2022 13:36:59 +0100, Laura Arjona Reina 
>  said:

> I've tried to update the https://www.debian.org/devel/website/todo page 
and did a partial rewrite, which is in:

> https://salsa.debian.org/webmaster-team/webwml/-/merge_requests/792

> please have a look, if there are no objections, I would commit that in 
some days.

I'm fine with these changes.

-- 
viele Grüße Thomas

Re: Website update

[Thomas Lange]

> On Tue, 19 Oct 2021 11:08:25 +0200, Hans Fredrik Nordhaug 
>  said:

> Just a question: Are translators taken into account when these 
improvements are made? It's
> beneficial to know if we need to review the complete text or just 
reformat. Some of the latest
> changes to the intro pages have made it hard to track the actual changes 
in the text (by looking at
> Git commits) ...

I'm sorry, but it was not possible to organise the git commits so the
changes for the translators were clear. I know it would have been
easier if we had more smaller commits so you could see which parts of
the text were new and which were only moved because of the newer
layout.

-- 
best regards Thomas

Re: Website update

[Thomas Lange]

Hi Paulo,

Heike is the external journalist who is helping us to improve the text
on our web pages. I got funding from our DPL to pay external help for
the web pages. I've created a list of web pages which I think are
important and need some love. Heike started with pages in /intro and
/devel but more pages will follow.

best regards Thomas

> On Sun, 10 Oct 2021 10:41:11 -0300, Paulo Henrique de Lima Santana 
>  said:

> Hi,
> I have seen some main web pages were updated by Heike Jurzik, and it was 
> a great work.
> Maybe I haven't seen previous message, but I'm curious: is Heike 
> reviewing all Debian web site?

Bug#364913: still valid?

[Thomas Lange]

Is this bug still valid or can we close this bug?
-- 
regards Thomas

Bug#222334: link to test page

[Thomas Lange]

We may add this link which shows your browser language settings

https://manytools.org/http-html-text/browser-language/

-- 
viele Grüße Thomas

Re: Debian website: authoritative copy of mirror list

[Thomas Lange]

> On Sat, 21 Aug 2021 09:38:42 +0200, Holger Wansing  
> said:

> I found this nonsens phrase:

> "The authoritative copy of the following list can always be found at: 
> https://www.debian.org/mirror/list.;

> So, the link points exactly to the page, on which this phrase is shown!

> Is there a real "authoritative copy" of the mirror list, you might want to
> link to here, or should I just remove that phrase completely?

The html version is generated from
https://salsa.debian.org/mirror-team/masterlist

I think our users (not developers) do not need the original file but
are happy with the web page. So remove this nonsens phrase.
-- 
viele Grüße Thomas