Re: Jigdo files wrong?

[Steve McIntyre]

Hi Marcelo,

On Mon, Mar 04, 2024 at 12:25:49PM -0300, Marcelo B. wrote:
>Hi guys, there seem to be a problem with AMD64's jigdo files of the weekly
>builds: they're all the same at 120 bytes in size.

ACK, thanks for reporting. I'll take a look.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
“Why do people find DNS so difficult? It’s just cache invalidation and
 naming things.”
   -– Jeff Waugh (https://twitter.com/jdub)

Re: jigdo DVD images> downloads fail

[Steve McIntyre]

[ Re-adding the CC to the mailing list; please respond there too ]
On Tue, Jun 08, 2021 at 01:17:16PM +0200, valentí juanola wrote:
>It works. I apologize for my ignorance. Thank you very much, Steve. Have a nice
>life.

OK, so it seems you need to repair your system's locale
information. I'd recommend you fix that too, as other programs may
well have similar problems if you don't. The following might help:

$ sudo apt install --reinstall locales

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
You raise the blade, you make the change... You re-arrange me 'til I'm sane...

Re: jigdo DVD images> downloads fail

[valentí juanola]

It works. I apologize for my ignorance. Thank you very much, Steve. Have a
nice life.

El dt., 8 de juny 2021, 13:03, Steve McIntyre  va
escriure:

> [ Re-adding the CC to the mailing list; please respond there too ]
>
> On Tue, Jun 08, 2021 at 12:32:18PM +, valentí juanola wrote:
> >AShFrtRm324fZ4pTjQvZRA  debian-10.9.0-amd64-DVD-1.template
>
> >
> >LANG=en_US.UTF-8
> >LANGUAGE=
> >LC_CTYPE="en_US.UTF-8"
> >LC_NUMERIC="en_US.UTF-8"
> >LC_TIME="en_US.UTF-8"
> >LC_COLLATE="en_US.UTF-8"
> >LC_MONETARY="en_US.UTF-8"
> >LC_MESSAGES=en_US.UTF-8
> >LC_PAPER="en_US.UTF-8"
> >LC_NAME="en_US.UTF-8"
> >LC_ADDRESS="en_US.UTF-8"
> >LC_TELEPHONE="en_US.UTF-8"
> >LC_MEASUREMENT="en_US.UTF-8"
> >LC_IDENTIFICATION="en_US.UTF-8"
> >LC_ALL=
>
> OK. That checksum matches what I have here, so the template download
> is fine. What happens if you run "export LC_ALL=C" before you start
> jigdo-lite? Your errors:
>
> >  jigdo-file: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt <
> >  (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))'
> >  failed.
> >  jigdo-file: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt <
> >  (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))'
> >  failed.
>
> suggest you might have something broken on your system in the
> localisation setup.
>
> --
> Steve McIntyre, Cambridge, UK.
> st...@einval.com
> Into the distance, a ribbon of black
> Stretched to the point of no turning back
>
>

Re: jigdo DVD images> downloads fail

[Steve McIntyre]

[ Re-adding the CC to the mailing list; please respond there too ]

On Tue, Jun 08, 2021 at 12:32:18PM +, valentí juanola wrote:
>AShFrtRm324fZ4pTjQvZRA  debian-10.9.0-amd64-DVD-1.template
>
>LANG=en_US.UTF-8
>LANGUAGE=
>LC_CTYPE="en_US.UTF-8"
>LC_NUMERIC="en_US.UTF-8"
>LC_TIME="en_US.UTF-8"
>LC_COLLATE="en_US.UTF-8"
>LC_MONETARY="en_US.UTF-8"
>LC_MESSAGES=en_US.UTF-8
>LC_PAPER="en_US.UTF-8"
>LC_NAME="en_US.UTF-8"
>LC_ADDRESS="en_US.UTF-8"
>LC_TELEPHONE="en_US.UTF-8"
>LC_MEASUREMENT="en_US.UTF-8"
>LC_IDENTIFICATION="en_US.UTF-8"
>LC_ALL=

OK. That checksum matches what I have here, so the template download
is fine. What happens if you run "export LC_ALL=C" before you start
jigdo-lite? Your errors:

>  jigdo-file: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt <
>  (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))'
>  failed.
>  jigdo-file: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt <
>  (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))'
>  failed.

suggest you might have something broken on your system in the
localisation setup.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
Into the distance, a ribbon of black
Stretched to the point of no turning back

Re: jigdo DVD images> downloads fail

[Steve McIntyre]

On Tue, Jun 08, 2021 at 11:57:48AM +0200, valentí juanola wrote:
>Dear Steve
>Thanks for your help. All Dvds give the same error message...

So what checksum do you get for the template file please?

Also, checking your error message again, what does the command
"locale" say on your system?

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
There's no sensation to compare with this
Suspended animation, A state of bliss

Re: jigdo DVD images> downloads fail

[valentí juanola]

Thanks, I'll check them out.

El dl., 7 de juny 2021, 13:46, Steve McIntyre  va
escriure:

> Hi Valentí,
>
> On Mon, Jun 07, 2021 at 01:30:32PM +, valentí juanola wrote:
> >[profile_ma]
> >
> > [profile_ma]
> >  Cannot download via jigdo because
> >
> >  Jigdo downloads fail at address
> >  https://cdimage.debian.org/debian-cd/current/amd64/jigdo-dvd/
> >
> >  file: debian-10.9.0-amd64-DVD-1.template
> >  from: https://laotzu.ftp.acc.umu.se(???)
> >
> >
> >  Debian mirror [ftp://ftp.de.debian.org/debian/]:
> >  jigdo-file: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt <
> >  (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))'
> >  failed.
> >  jigdo-file: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt <
> >  (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))'
> >  failed.
> >  Error - template checksum mismatch!
> >  The .template file does not belong to the .jigdo file
> >
> >  Please help. Thank you. Val.
>
> Hmmm. That jigdo and template file work just fine here:
>
> $ jigdo-lite
> https://cdimage.debian.org/debian-cd/current/amd64/jigdo-dvd/debian-10.9.0-amd64-DVD-1.jigdo
> ...
> Downloading .template file
> --2021-06-07 12:40:41--
> https://cdimage.debian.org/debian-cd/current/amd64/jigdo-dvd/debian-10.9.0-amd64-DVD-1.template
> Resolving cdimage.debian.org (cdimage.debian.org)... 2001:6b0:19::165,
> 2001:6b0:19::163, 2001:6b0:19::173, ...
> Connecting to cdimage.debian.org (cdimage.debian.org)|2001:6b0:19::165|:443...
> connected.
> HTTP request sent, awaiting response... 302 Found
> Location:
> https://laotzu.ftp.acc.umu.se/debian-cd/current/amd64/jigdo-dvd/debian-10.9.0-amd64-DVD-1.template
> [following]
> --2021-06-07 12:40:41--
> https://laotzu.ftp.acc.umu.se/debian-cd/current/amd64/jigdo-dvd/debian-10.9.0-amd64-DVD-1.template
> Resolving laotzu.ftp.acc.umu.se (laotzu.ftp.acc.umu.se)...
> 2001:6b0:19::166, 194.71.11.166
> Connecting to laotzu.ftp.acc.umu.se 
> (laotzu.ftp.acc.umu.se)|2001:6b0:19::166|:443...
> connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 37711251 (36M)
> Saving to: ‘debian-10.9.0-amd64-DVD-1.template’
> ...
> 2021-06-07 12:40:48 (5.60 MB/s) - ‘debian-10.9.0-amd64-DVD-1.template’
> saved [37711251/37711251]
>
> What checksum do you have on debian-10.9.0-amd64-DVD-1.template?
>
> $ sha256sum debian-10.9.0-amd64-DVD-1.template
> 2754a430f357f782fede008a83cc72f281ce6150e4b84219b3ef8a142e875768
> debian-10.9.0-amd64-DVD-1.template
>
> --
> Steve McIntyre, Cambridge, UK.
> st...@einval.com
> “Why do people find DNS so difficult? It’s just cache invalidation and
>  naming things.”
>-– Jeff Waugh (https://twitter.com/jdub)
>
>

Re: jigdo DVD images> downloads fail

[Steve McIntyre]

Hi Valentí,

On Mon, Jun 07, 2021 at 01:30:32PM +, valentí juanola wrote:
>[profile_ma]
>
> [profile_ma]
>  Cannot download via jigdo because
>
>  Jigdo downloads fail at address
>  https://cdimage.debian.org/debian-cd/current/amd64/jigdo-dvd/
>
>  file: debian-10.9.0-amd64-DVD-1.template
>  from: https://laotzu.ftp.acc.umu.se    (???)
>
>
>  Debian mirror [ftp://ftp.de.debian.org/debian/]:
>  jigdo-file: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt <
>  (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))'
>  failed.
>  jigdo-file: loadlocale.c:130: _nl_intern_locale_data: Assertion `cnt <
>  (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))'
>  failed.
>  Error - template checksum mismatch!
>  The .template file does not belong to the .jigdo file
>
>  Please help. Thank you. Val.

Hmmm. That jigdo and template file work just fine here:

$ jigdo-lite 
https://cdimage.debian.org/debian-cd/current/amd64/jigdo-dvd/debian-10.9.0-amd64-DVD-1.jigdo
...
Downloading .template file
--2021-06-07 12:40:41--  
https://cdimage.debian.org/debian-cd/current/amd64/jigdo-dvd/debian-10.9.0-amd64-DVD-1.template
Resolving cdimage.debian.org (cdimage.debian.org)... 2001:6b0:19::165, 
2001:6b0:19::163, 2001:6b0:19::173, ...
Connecting to cdimage.debian.org (cdimage.debian.org)|2001:6b0:19::165|:443... 
connected.
HTTP request sent, awaiting response... 302 Found
Location: 
https://laotzu.ftp.acc.umu.se/debian-cd/current/amd64/jigdo-dvd/debian-10.9.0-amd64-DVD-1.template
 [following]
--2021-06-07 12:40:41--  
https://laotzu.ftp.acc.umu.se/debian-cd/current/amd64/jigdo-dvd/debian-10.9.0-amd64-DVD-1.template
Resolving laotzu.ftp.acc.umu.se (laotzu.ftp.acc.umu.se)... 2001:6b0:19::166, 
194.71.11.166
Connecting to laotzu.ftp.acc.umu.se 
(laotzu.ftp.acc.umu.se)|2001:6b0:19::166|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 37711251 (36M)
Saving to: ‘debian-10.9.0-amd64-DVD-1.template’
...
2021-06-07 12:40:48 (5.60 MB/s) - ‘debian-10.9.0-amd64-DVD-1.template’ saved 
[37711251/37711251]

What checksum do you have on debian-10.9.0-amd64-DVD-1.template?

$ sha256sum debian-10.9.0-amd64-DVD-1.template
2754a430f357f782fede008a83cc72f281ce6150e4b84219b3ef8a142e875768  
debian-10.9.0-amd64-DVD-1.template

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
“Why do people find DNS so difficult? It’s just cache invalidation and
 naming things.”
   -– Jeff Waugh (https://twitter.com/jdub)

Re: jigdo Problems while processing https://cdimage.debian.org/cdimage/archive/9.12.0/armhf/jigdo-dvd/debian-update-9.12.0-armhf-DVD-1.jigdo ..

[Steve McIntyre]

Hi Bernd!

On Wed, Feb 19, 2020 at 12:04:29PM +0100, Bernd Oelker (BeOP-IT) wrote:
>Hallo List,
>
>while update all debian dvd'2 from debian-9.11 to debian 9.12
>(arch=armhf) everything worked as expected for all
>
>debian-9.12.0-armhf-DVD-[1..13].jigdo files, the
>
>debian-update-9.12.0-armhf-DVD-[1.2].jigdo failed, because the remaining
>*.deb files are not found ??
>
>URL's ftp2.de.debian.org ...
>
>for example:
>
>--2020-02-19 09:14:04-- 
>ftp://ftp2.de.debian.org/pool/main/f/firefox-esr/iceweasel-l10n-vi_60.9.0esr-1~deb9u1_all.deb
>   =>
>»./debian-update-9.12.0-armhf-DVD-1.iso.tmpdir/ftp2.de.debian.org/pool/main/f/firefox-esr/iceweasel-l10n-vi_60.9.0esr-1~deb9u1_all.deb«
>Verbindungsaufbau zu ftp2.de.debian.org
>(ftp2.de.debian.org)|137.226.34.46|:21 … verbunden.
>Anmelden als anonymous … Angemeldet!
>==> SYST ... fertig.    ==> PWD ... fertig.
>==> TYPE I ... fertig.  ==> CWD (1) /pool/main/f/firefox-esr ...
>Das Verzeichnis »pool/main/f/firefox-esr« existiert nicht.
>
>This only happens for the update-DVD's ??

As mentioned in IRC, I think this is a problem with how you're driving
jigdo. The URL for the missing file should be

  
ftp://ftp2.de.debian.org/debian/pool/main/f/firefox-esr/iceweasel-l10n-vi_60.9.0esr-1~deb9u1_all.deb

(note the "debian/" ahead of "pool/". This *suggests* you've told
jigdo to use the wrong URL for your mirror, and missed off the
"debian/" component of the path.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
"When C++ is your hammer, everything looks like a thumb." -- Steven M. Haflich

Re: jigdo-file: Does not report package rejections because checksum mismatch

[Thomas Schmitt]

Hi,

i wrote:
> > us.cdimage.debian.org is a quick one.

Nicholas Geovanis wrote:
> From which location? Germany?

Ja.


How about this final message in case that files are missing and that
mismatching downloads were detected ?
(The mismatches shown are fake and recorded twice to get more than 2.)

== begin of example
-
Aaargh - 1 files remain missing. This should not happen!
4 download attempts yielded files with mismatching MD5 checksums:

   
http://archive.debian.org/debian/pool/main/p/partman-reiserfs/partman-reiserfs_50_all.udeb
   
http://archive.debian.org/debian/pool/main/p/partman-reiserfs/partman-reiserfs_50_all.udeb
   
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/p/partman-reiserfs/partman-reiserfs_50_all.udeb
   ... the WARNING messages of this run report more ...

After a retry with the same mirror consider to search the web
for the names of remaining mismatching packages. As mirror name
use the found URL up to the "/" before the directory name "pool".

Press Return to retry downloading the missing files.
Press Ctrl-C to abort. (If you re-run jigdo-lite later, it will
resume from here, the downloaded data is not lost if you press
Ctrl-C now.)
:
== end of example

Is the advise understandable ?
I propose one repetition to remove any warnings from the user defined mirror
which were resolved by the fallback mirror. In the second run only messages
about the problematic files should emerge, hopefully making it easier to
spot the checksum warnings.

The list of URLs must be restricted to 3 and the usual messages must be
omitted in order to keep the text below 24 lines if the URLs are longer
than two lines.

The usual message appears if no mismatches were detected but still files
are missing.

I changed the old message
  Aaargh - 1 files could not be downloaded. This should not
  happen! ...
in both cases to
  Aaargh - 1 files remain missing. This should not happen!
because it matches better the spectrum of potential problem causes.


--- /usr/bin/jigdo-lite.sid 2017-12-28 14:20:23.882643023 +0100
+++ /home/thomas/projekte/jigdo_dir/jigdo-lite.sid.with_md5_check   
2017-12-29 20:09:34.055048360 +0100
@@ -29,6 +29,13 @@ else
   windows=false
   nl='\n'
 fi
+
+# Counter of MD5 mismatches after download
+fileMismatchCount=0
+# Short list of URLs which yielded mismatch
+fileMismatchList=""
+# Very few mismatching URLs shall be shown at the end. They can be long.
+fileMismatchMaxRec=3
 #__
 
 # read with readline, only if running bash >=2.03 (-e gives error on POSIX)
@@ -75,10 +82,84 @@ fetch() {
 }
 #__
 
-# Given URLs, fetch them into $imageTmp, then merge them into image
+# DEVELOPMENT TEST:
+# Set to a package name of the ISO to get simulated MD5 mismatch
+# simulateMD5Mismatch="partman-reiserfs_50_all.udeb"
+simulateMD5Mismatch="NOT_A_PACKAGE_NAME"
+
+# Given URLs and MD5s, fetch them into $imageTmp and verify,
+# then merge them into image
 fetchAndMerge() {
+
+  # The other arguments are URLs in the same sequence as the words in md5List
+  md5List="$1"
+  shift 1
+
   if test "$#" -eq 0; then return 0; fi
   fetch --force-directories --directory-prefix="$imageTmp" -- "$@"
+
+  # Try to verify downloaded files
+  for md5 in $md5List
+  do
+url="$1"
+shift 1
+test "$md5" = ".no.MD5.known." && continue
+
+# Simulated MD5 mismatch
+if echo "$url" | grep '/'"$simulateMD5Mismatch"'$' >/dev/null 2>&1
+then
+  echo "DEVELOPMENT TEST: Faking a checksum mismatch with package 
$simulateMD5Mismatch" >&2
+  md5="*INVALIDATED*CHECKSUM*"
+fi
+
+# Alternative proposals by Philip Hands:
+# localPath="$imageTmp"/`echo "$url" | sed -e 
's,^\(https\?\|ftp\|file\)://,,i'`
+# localPath="$imageTmp/${url#[[:alpha:]]*://}"
+
+localPath="$imageTmp"/`echo "$url" | \
+sed -e 's/^[hH][tT][tT][pP]:\/\///' \
+-e 's/^[hH][tT][tT][pP][sS]:\/\///' \
+-e 's/^[fF][tT][pP]:\/\///' \
+-e 's/^[fF][iI][lL][eE]:\/\///'`
+
+if test ! -e "$localPath"
+then
+  # Maybe the file was downloaded but above guess was wrong
+  baseName=`basename "$url"`
+  localPath=`find "$imageTmp" -name "$baseName" | head -1`
+fi
+
+if test -n "$localPath" -a -e "$localPath"
+then
+  fileMD5=`$jigdoFile md5sum --report=quiet "$localPath" | sed 's/ .*$//'`
+  if test "$md5" != "$fileMD5"
+  then
+echo >&2
+echo "WARNING: Downloaded file does not match expected MD5:" >&2
+echo " $url" >&2
+echo " 

Re: jigdo-file: Does not report package rejections because checksum mismatch

[Nicholas Geovanis]

On Fri, Dec 29, 2017 at 5:35 AM, Thomas Schmitt  wrote:
> I wrote:
>> > > >   sed -e 's/^[hH][tT][tT][pP]:\/\///' \
>> > > >... more -e for https, ftp, and file ...
>
> Philip Hands wrote:
>> > >   sed -e 's,^\(https\?\|ftp\|file\)://,,i'
>> > > ...
>> > >  "$imageTmp/${url#[[:alpha:]]*://}"
>
>> > Are these widely portable enough ?
>
>> [Philip Hands analyzed portability with positive result]
>
> I still think that the long explicit sed is clearer.

FWIW I agree.

And with respect to the new awk dependency mentioned earlier, it's a simple
expression and awk tempts me with that all the time because it's so much easier.
Just you know, be advised that on say Debian 8.9 it's gawk while on Ubuntu 17.10
it's mawk, etc. Debian 9.2 is still gawk.

> 
> The effective throughput of roughly 1.5 to 2.5 MB/s is still much slower
> than wget's speed report of about 5.5 MB/s.
> I tried with 100 files per run of wget and "jigdo-file make-image".
> No significant difference to see. It's all about mirror server latency
> with each single file, i guess.
> us.cdimage.debian.org is a quick one.

>From which location? Germany?
I find it a bit slower than average from here inside the US, but I
have large, local
mirrors. And at long last, one of my own is budgeted in 2018. You betcha ;-)

> I wrote:
>> > > > fileMD5=`$jigdoFile md5 "$localPath" 2>/dev/null | awk '{print $1}'`
>
> Philip Hands wrote:
>> `$jigdoFile md5sum --report=quiet "$localPath" | sed 's/ .*$//'`
>
> I'll take this one.

Oh, OK. Yeah, better.

> Next development step will be to issue a correct "Aaargh" message and to
> tell at least some of the mismatching files in that message.
> Have a nice day :)

Vielen Dank Thomas.

> Thomas
>

Re: jigdo-file: Does not report package rejections because checksum mismatch

[Thomas Schmitt]

Hi,

I wrote:
> > > >   sed -e 's/^[hH][tT][tT][pP]:\/\///' \
> > > >... more -e for https, ftp, and file ...

Philip Hands wrote:
> > >   sed -e 's,^\(https\?\|ftp\|file\)://,,i'
> > > ...
> > >  "$imageTmp/${url#[[:alpha:]]*://}"

> > Are these widely portable enough ?

> [Philip Hands analyzed portability with positive result]

I still think that the long explicit sed is clearer. But in the end it will
be up to Steve to decide which one to use.
I tested both proposals of yours and have put them as comments into my
evolving changeset.

The importance of this expression has decreased by my decision to run "find"
if the guessed local path does not lead to an existing file:

localPath=... guessed from URL ...

if test ! -e "$localPath"
then
  # Maybe above guess was wrong
  baseName=`basename "$url"`
  localPath=`find "$imageTmp" -name "$baseName" | head -1`
fi

if test -n "$localPath" -a -e "$localPath"
then
   ... checksum verification ...

The use of "head" and "find" will be new in the script. But the increased
ruggedness makes it worthwhile in my opinion.

I made mini benchmarks with guessed names and found names. No significant
differences were to see.
(The tree is really small because fetchAndMerge() deletes it when the 10
 files are processed.)

The effective throughput of roughly 1.5 to 2.5 MB/s is still much slower
than wget's speed report of about 5.5 MB/s.
I tried with 100 files per run of wget and "jigdo-file make-image".
No significant difference to see. It's all about mirror server latency
with each single file, i guess.
us.cdimage.debian.org is a quick one.


I wrote:
> > > > fileMD5=`$jigdoFile md5 "$localPath" 2>/dev/null | awk '{print $1}'`

Philip Hands wrote:
> `$jigdoFile md5sum --report=quiet "$localPath" | sed 's/ .*$//'`

I'll take this one.


Next development step will be to issue a correct "Aaargh" message and to
tell at least some of the mismatching files in that message.


Have a nice day :)

Thomas

Re: jigdo-file: Does not report package rejections because checksum mismatch

[Philip Hands]

On Thu, 28 Dec 2017, "Thomas Schmitt"  wrote:
> Hi,
>
> first a correction of my proposal:
>
> The else-case with
>   echo "WARNING: File not found after download:" >&2
> is not good.
> It floods the log if one uses a mirror with few matching files.
> Not finding a file after the download is a normal situation.
>
> ---
>
> I wrote:
>> >sed -e 's/^[hH][tT][tT][pP]:\/\///' \
>> >-e 's/^[hH][tT][tT][pP][sS]:\/\///' \
>> >-e 's/^[fF][tT][pP]:\/\///' \
>> >-e 's/^[fF][iI][lL][eE]:\/\///'`
>
> Philip Hands wrote:
>>   sed -e 's,^\(https\?\|ftp\|file\)://,,i'
>> ...
>>   "$imageTmp/${url#[[:alpha:]]*://}"
>
> Are these widely portable enough ?

the , rather than / feature is already in use in the script (except that
its using s%%%).

\( \) is already in use, and AFAIK \| has been there for as long

\? _might_ be a bit later as a feature, in which case one could add
\|https, but then again isURI() doesn't match https: anyway

The i flag is a GNU extension, so is probably not that portable, so one
could go for \(http\|HTTP\|...

For the shell, I suspect that [[:alpha:]] is an innovation from the
90's, so one could play it safe (well, except that it might break with
odd codings) with [a-zA-Z]. posh doesn't seem to know about [:alpha:]
for instance.

posh does know about the ${ # } thing, but that wasn't in Solaris SVR4
shell AFAIK.

> Mine can be justified by S.R.Bourne's "The Unix System", i guess,
> and it is coordinated with function isURI.
>
> Well, my scruples are mainly about what wget guarantees to use as
> local disk path. I understand that jigdo-file would be quite tolerant
> as long as the file is somewhere in the "$imageTmp" tree.
> Maybe i should invest a "find" run in case of missing file. The tree is small.
>
>
> I wrote:
>> > fileMD5=`$jigdoFile md5 "$localPath" 2>/dev/null | awk '{print $1}'`
>
>> The way it's done elsewhere in the script (which I happen to think is
>> pretty horrible, but that's what is already there) is using set, thus:
>>   set -- `$jigdoFile md5sum --report=quiet "$localPath"`
>
> Option "--report=quiet" instead of "2>/dev/null" is a good point.
>
> One would have to wrap the "set --" into a sub-shell, because fetchAndMerge
> already tampers with its own arguments.
> Like:
>   answer=`$jigdoFile md5sum --report=quiet "$localPath"`
>   fileMD5=`(set -- $answer ; echo "$1")`
> Now that's really ugly.

This seems preferable, and avoids new dependencies:

  `$jigdoFile md5sum --report=quiet "$localPath" | sed 's/ .*$//'`

> If direct objections emerge against "awk", i'd consider some helper
> function which echos "$1".
>
>
>> I also happen to think that using `` rather than $() is pretty horrible
>> in this day and age, but that's what's currently there throughout the
>
> Yep. Not to speak of the headless camelBack variable names.
>
> I strive to be minimally intrusive for the purpose and to be as
> conservative as in an autotools script.

Fair enough.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Re: jigdo-file: Does not report package rejections because checksum mismatch

[Thomas Schmitt]

Hi,

first a correction of my proposal:

The else-case with
  echo "WARNING: File not found after download:" >&2
is not good.
It floods the log if one uses a mirror with few matching files.
Not finding a file after the download is a normal situation.

---

I wrote:
> >sed -e 's/^[hH][tT][tT][pP]:\/\///' \
> >-e 's/^[hH][tT][tT][pP][sS]:\/\///' \
> >-e 's/^[fF][tT][pP]:\/\///' \
> >-e 's/^[fF][iI][lL][eE]:\/\///'`

Philip Hands wrote:
>   sed -e 's,^\(https\?\|ftp\|file\)://,,i'
> ...
>   "$imageTmp/${url#[[:alpha:]]*://}"

Are these widely portable enough ?
Mine can be justified by S.R.Bourne's "The Unix System", i guess,
and it is coordinated with function isURI.

Well, my scruples are mainly about what wget guarantees to use as
local disk path. I understand that jigdo-file would be quite tolerant
as long as the file is somewhere in the "$imageTmp" tree.
Maybe i should invest a "find" run in case of missing file. The tree is small.


I wrote:
> > fileMD5=`$jigdoFile md5 "$localPath" 2>/dev/null | awk '{print $1}'`

> The way it's done elsewhere in the script (which I happen to think is
> pretty horrible, but that's what is already there) is using set, thus:
>   set -- `$jigdoFile md5sum --report=quiet "$localPath"`

Option "--report=quiet" instead of "2>/dev/null" is a good point.

One would have to wrap the "set --" into a sub-shell, because fetchAndMerge
already tampers with its own arguments.
Like:
  answer=`$jigdoFile md5sum --report=quiet "$localPath"`
  fileMD5=`(set -- $answer ; echo "$1")`
Now that's really ugly.

If direct objections emerge against "awk", i'd consider some helper
function which echos "$1".


> I also happen to think that using `` rather than $() is pretty horrible
> in this day and age, but that's what's currently there throughout the

Yep. Not to speak of the headless camelBack variable names.

I strive to be minimally intrusive for the purpose and to be as
conservative as in an autotools script.


An alternative to changing the code would still be to tell the user with
the "Aaargh" text that repeated download and subsequent "Aaargh" could
indicate damaged files on the mirror. In this case the user shall search
the web for other mirrors which offer the repeatedly downloaded packages.

But that would be embarrassing for the involved programmers.
(Having script jigdo-lite instead of doing the job inside jigdo-file is
 also not overly glorious ...)


Have a nice day :)

Thomas

Re: jigdo-file: Does not report package rejections because checksum mismatch

[Philip Hands]

On Thu, 28 Dec 2017, "Thomas Schmitt"  wrote:
...
> Especially in need of attention:
>
> - I could not find a clear description how "wget" determines the local
>   file paths from URLs and option --directory-prefix="$imageTmp".
>   My current conversion from URL to path is purely heuristic therefore:
>
> localPath="$imageTmp"/`echo "$url" | \
>sed -e 's/^[hH][tT][tT][pP]:\/\///' \
>-e 's/^[hH][tT][tT][pP][sS]:\/\///' \
>-e 's/^[fF][tT][pP]:\/\///' \
>-e 's/^[fF][iI][lL][eE]:\/\///'`

A rather less laboured way of getting the same effect with sed would be:

  sed -e 's,^\(https\?\|ftp\|file\)://,,i'

[ Things to note about that:
 s,,, in place of s/// means that no escaping of / is needed
 the 'i' flag at the end makes the match case insensitive
 s\? means match zero or one 's'
]

However, I doubt that it's important to worry about the potential for
unexpectedly removing a prefix of e.g. cdrom:// or ://, in which case
you could dispense with sed and instead do this:

  localpath="$imageTmp/${url#[[:alpha:]]*://}"

> - I introduced a dependency on "awk", which was not used in jigdo-lite
>   before. The task is to obtain the first word of jigdo-file's output:
>
> fileMD5=`$jigdoFile md5 "$localPath" 2>/dev/null | awk '{print $1}'`

The way it's done elsewhere in the script (which I happen to think is
pretty horrible, but that's what is already there) is using set, thus:

  set -- `$jigdoFile md5sum --report=quiet "$localPath"`

which leaves the value that you are after in $1.

I also happen to think that using `` rather than $() is pretty horrible
in this day and age, but that's what's currently there throughout the
script, so I guess one should stick with that, or fix it everywhere.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Re: jigdo-file: Does not report package rejections because checksum mismatch

[Thomas Schmitt]

Hi,

i Cc: debian-cd with this follow-up to bug 884526 in the hope to get
some review for the endeavor to detect damaged downloaded package files
during a run of jigdo-lite.

Some disputable aspects remain (plus a possible bug in current jigdo-lite,
which will vanish by my proposal). I put them behind my changeset, which is
based on Sid of today.


How it works:

Currently and in my proposal jigdo-lite asks jigdo-file for a list of files
yet missing in the emerging image. It downloads them in groups of 10 and
submits each group to jigdo-file for grafting them into the image.
The list consists of text blocks of the form:
  URL of package
  Alternative URL of same package
  ... more alternative URLs ...
  MD5Sum:...
  
One of the URLs from each block gets picked by a loop with variable "pass",
but currently jigdo-file ignores the MD5Sum lines.

My proposal collects the MD5s and submits them to function fetchAndMerge
which currently only gets the 10 URLs.
I had to re-arrange the entrails of the list reading loop in imageDownload.
It currently ends when the 10th URL is found and thus would not have yet
reached the 10th checksum line. My proposal uses the empty line as trigger
to append the picked URL and the MD5 to the argument list.

Tested with Jessie's jigdo-file and these three input lines:

  
http://cdimage.debian.org/mirror/cdimage/archive/6.0.7/amd64/jigdo-cd/debian-6.0.7-amd64-businesscard.jigdo
  
  http://us.cdimage.debian.org/cdimage/snapshot/Debian/

--- /usr/bin/jigdo-lite.sid 2017-12-28 14:20:23.882643023 +0100
+++ /home/thomas/projekte/jigdo_dir/jigdo-lite.sid.with_md5_check   
2017-12-28 15:12:37.738654856 +0100
@@ -75,10 +75,61 @@ fetch() {
 }
 #__
 
-# Given URLs, fetch them into $imageTmp, then merge them into image
+# Simulated MD5 mismatch:
+simulateMD5Mismatch="partman-reiserfs_50_all.udeb"
+# simulateMD5Mismatch="NOT_A_PACKAGE_NAME"
+
+# Given URLs and MD5s, fetch them into $imageTmp and verify,
+# then merge them into image
 fetchAndMerge() {
+
+  # The other arguments are URLs in the same sequence as the words in md5List
+  md5List="$1"
+  shift 1
+
   if test "$#" -eq 0; then return 0; fi
   fetch --force-directories --directory-prefix="$imageTmp" -- "$@"
+
+  # Try to verify downloaded files
+  for md5 in $md5List
+  do
+url="$1"
+shift 1
+test "$md5" = ".no.MD5.known." && continue
+
+# Simulated MD5 mismatch
+if echo "$url" | grep '/'"$simulateMD5Mismatch"'$' >/dev/null 2>&1
+then
+  echo "ATTENTION : Faking a checksum mismatch with package 
$simulateMD5Mismatch" >&2
+  md5="*INVALIDATED*CHECKSUM*"
+fi
+
+localPath="$imageTmp"/`echo "$url" | \
+sed -e 's/^[hH][tT][tT][pP]:\/\///' \
+-e 's/^[hH][tT][tT][pP][sS]:\/\///' \
+-e 's/^[fF][tT][pP]:\/\///' \
+-e 's/^[fF][iI][lL][eE]:\/\///'`
+if test -e "$localPath"
+then
+  fileMD5=`$jigdoFile md5 "$localPath" 2>/dev/null | awk '{print $1}'`
+  if test "$md5" != "$fileMD5"
+  then
+echo >&2
+echo "WARNING: Downloaded file does not match expected MD5:" >&2
+echo " $url" >&2
+echo " $localPath" >&2
+echo " expected: $md5 | $fileMD5 :downloaded" >&2
+echo >&2
+  fi
+else
+  echo >&2
+  echo "WARNING: File not fot found after download:" >&2
+  echo " $url" >&2
+  echo " $localPath" >&2
+  echo >&2
+fi
+  done
+
   # Merge into the image
   $jigdoFile $jigdoOpts --no-cache make-image --image="$image" \
 --jigdo="$jigdoF" --template="$template" "$imageTmp"
@@ -596,31 +647,56 @@ imageDownload() {
 for pass in x xx xxx  x xx xxx ; do
   $jigdoFile print-missing-all --image="$image" --jigdo="$jigdoF" \
 --template="$template" $jigdoOpts $uriOpts \
-  | egrep -i '^(http:|ftp:|$)' >"$list"
+   >"$list"
+  # This counts the empty lines
   missingCount=`egrep '^$' <"$list" | wc -l | sed -e 's/ *//g'`
   # Accumulate URLs in $@, pass them to fetchAndMerge in batches
   shift "$#" # Solaris /bin/sh doesn't understand "set --"
   count=""
   exec 3<"$list"
+  useUrl="."
+  md5=".no.MD5.known."
+  md5List=""
   while $readLine url <&3; do
 count="x$count"
-if strEmpty "$url"; then count=""; continue; fi
-if test "$count" != "$pass"; then continue; fi
-if $noMorePasses; then
-  hrule
-  echo "$missingCount files not found in previous pass, trying"
-  echo "alternative download locations:"
-  echo
-fi
-noMorePasses=false
-set -- "$@" "$url"
-if test "$#" -ge "$filesPerFetch"; then
-  if fetchAndMerge "$@"; then true; 

Re: jigdo-lite for windows

[Thomas Schmitt]

Hi,

i asked Florent Larose to move the thread over to
  debian-u...@lists.debian.org
for now.


Have a nice day :)

Thomas

Re: jigdo-lite for windows

[Thomas Schmitt]

Hi,

Try asking at mailing list

  debian-user-fre...@lists.debian.org 

See
  https://lists.debian.org/debian-user-french/

Ask for help with
  https://www.debian.org/CD/jigdo-cd/#how


As for sarge-amd64-1.jigdo ... was there Jigdo in Debian 3 ten
years ago ?

> la derniere distribution debian pour amd64.

The newest stable release is Debian 8 "Jessie".

  http://cdimage.debian.org/debian-cd/8.3.0/amd64/

If the installation target machine has internet accesss, consider
to use
  
http://cdimage.debian.org/debian-cd/8.3.0/amd64/iso-cd/debian-8.3.0-amd64-netinst.iso
instead of
  http://cdimage.debian.org/debian-cd/8.3.0/amd64/jigdo-dvd/


Have a nice day :)

Thomas

Re: jigdo-lite for windows

[Thomas Schmitt]

Hi,

i can only test on Debian GNU/Linux, not on MS-Windows.
I ran

  $ jigdo-lite

and was asked for the URL

  To start a new download, enter URL of .jigdo file.
  ...
  jigdo:

I gave the one of CD 1 to keep the test small:

  
http://cdimage.debian.org/debian-cd/8.3.0/amd64/jigdo-cd/debian-8.3.0-amd64-CD-1.jigdo

You will probably begin by

  
http://cdimage.debian.org/debian-cd/8.3.0/amd64/jigdo-dvd/debian-8.3.0-amd64-DVD-1.jigdo

It then showed lots of messages and asked

  Alternatively, just press enter if you want to start downloading
  the remaining files.
  Files to scan:

So i just pressed enter.
It asked me for a Debian mirror

  Debian mirror [http://ftp.de.debian.org/debian/]: 

I just pressed enter again.
Now i got internet traffic. jigdo-lite fetched the Debian packages which
fit into the holes of the .template file.
(At this point i have to re-iterate that netinst .iso CDs would
 fetch only what is needed when it is needed. So if internet is
 available at installation time in good quality ...)

After a few minutes the CD image was complete

  OK: Checksums match, image is good!   

I am mistrusting and determine the MD5 checksum of the result file

  $ md5sum debian-8.3.0-amd64-CD-1.iso
  a3e289b886c43150431b05b091c51ad8  debian-8.3.0-amd64-CD-1.iso

This matches the first line in
  http://cdimage.debian.org/debian-cd/8.3.0/amd64/jigdo-cd/MD5SUMS


To get the complete collection of all Debian packages you will have
to do this with each of the 13 DVD .jigdo files in
   http://cdimage.debian.org/debian-cd/8.3.0/amd64/jigdo-dvd/
(The two debian-update DVDs are not needed, afaik.)
With Blu-ray, it would be only 3 media images.


Have a nice day :)

Thomas

Re: Jigdo is NOT an option

[Franco Martelli]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Il 26/01/2015 01:12, Cyril Brulebois wrote:

 It looks to me like you might be hitting some network-specific
 issues. You probably want to provide with more information about
 your setup, your network, and your link(s) to the internets. So far
 as I can tell you're the only one complaining about
 cdimage.debian.org.
 
 I haven't seen anything like a ping, traceroute, or nmap capture so
 far, which would probably a better way to introduce yourself than
 complaining about this *very problematic* server without any
 facts.
 
 Can we please switch to a constructive approach?
Yesterday me too had trouble attempting to connect to
cdimage.debian.org I download the iso using a bash shell script and
the command:
wget -O - http://cdimage.debian.org/debian-cd/7.8.0/amd64/jigdo-dvd/
fails reporting that no data was send from the server. Changing ISP
for me did the trick... it looks like that cdimage.debian.org is fine,
today my script downloads the 10 isos of Wheezy 7.8.0 regularly and
now all works nicely.
HTH
Best regards

- -- 
Franco Martelli

  * Inglese - rilevata
  * Inglese
  * Italiano

  * Inglese
  * Italiano

 javascript:void(0);
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)

iQEbBAEBAgAGBQJUxpwuAAoJEFM/ma7n+T+7MwQH93OoB/DdRxs+QHhFJTyNqVOJ
ybSFCblKZUWEbrlVVE4aZpQn+3ihiogmHJup7JfOqSFQSyI9j/accji7SXxRyLdq
XimG+Fjn/Pu9BRfyjeigpz+G+RokVmoHBXswseffWT+6SrlhTv5Z/mJa4zH1ffAO
RsvcfSfUmHyXzpFzUZRcbqXf1qfjHIlpbqpLNkBZPyDi0VEEsCCE+Nn5BS5Rj+Oy
ugSwat85fWaplYl44sBjPH+xGUOgcxYjnfgZjFCopdPaaphDLkRVtE2NJslPywZz
wLINXFGUuIPn2dXecmPkQJmDcSnPYGYDm4QhYRsvAvET1ZIuY37CID52id3dHA==
=RwWu
-END PGP SIGNATURE-


-- 
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/54c69c2e.5090...@gmail.com

Re: Jigdo is NOT an option

[Cyril Brulebois]

Hello Heiko,

Heiko Schroeder hei...@evb-1.od.shuttle.de (2015-01-22):
 Hi Steve,
 
 
 I've not seen many reports of problems with cdimage.debian.org - what
 Although www.debian.org is reachable  cdimage.debian.org is not.  There is
 no chance to install the i386-jigdo-file an I need a full installation for
 tomorrow for a school ;-(.

It looks to me like you might be hitting some network-specific issues.
You probably want to provide with more information about your setup,
your network, and your link(s) to the internets. So far as I can tell
you're the only one complaining about cdimage.debian.org.

I haven't seen anything like a ping, traceroute, or nmap capture so far,
which would probably a better way to introduce yourself than complaining
about this *very problematic* server without any facts.

Can we please switch to a constructive approach?

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: Jigdo is NOT an option

[Heiko Schroeder]

Hi Matthias,


What's problematic with it?
The great problem seems to be cdimage.debian.org.  It is not  
reachable.   Even not for netinst-Images.


Best regards
Heiko Schroeder


--
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/20150122165536.horde.y6rv0yeqljfbb5bmq99n...@mail.shuttle.de

Re: Jigdo is NOT an option

[Heiko Schroeder]

Hi Steve,



I've not seen many reports of problems with cdimage.debian.org - what
Although www.debian.org is reachable  cdimage.debian.org is not.   
There is no chance to install the i386-jigdo-file an I need a full  
installation for tomorrow for a school ;-(.


Best regards
Heiko Schroeder


--
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/20150122165144.horde.bm2js0a37pnafnyo5nda...@mail.shuttle.de

Re: Jigdo is NOT an option

[Mattias Wadenstein]

On Thu, 22 Jan 2015, Heiko Schroeder wrote:


Dear team,

since the server cdimage.debian.org is *very problematic*, the download with 
jigdo is by far not an option for sysadmins to install the Debian System. 
Since Debian is very good it is also a very annoying fact.


What's problematic with it?

And if there is something problematic with it, does that also apply to the 
mirrors?


The jigdo option should only be needed for antique releases or odd 
architectures without network connectivity.


/Mattias Wadenstein


--
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/alpine.deb.2.10.1501221636130.1...@hirohito.acc.umu.se

Re: Jigdo is NOT an option

[Heiko Schroeder]

I have no problems reaching it, so I can't really tell what your  
problem with it is.

Yes, I understand.  But I have really massive problems; till now.



http://ftp.de.debian.org/debian-cd/

That is splendid!  Great!  Thanks a lot!

Best regards
Heiko


--
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/20150122171454.horde.g7z1komajquwz66zjorb...@mail.shuttle.de

Re: Jigdo is NOT an option

[Steve McIntyre]

On Thu, Jan 22, 2015 at 04:25:16PM +0100, Heiko Schroeder wrote:
Dear team,

since the server cdimage.debian.org is *very problematic*, the
download with jigdo is by far not an option for sysadmins to install
the Debian System.  Since Debian is very good it is also a very
annoying fact.

Hi Heiko,

I've not seen many reports of problems with cdimage.debian.org - what
exactly is not working for you, please?

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
You raise the blade, you make the change... You re-arrange me 'til I'm sane...


-- 
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20150122153928.gy27...@einval.com

Re: Jigdo is NOT an option

[Mattias Wadenstein]

On Thu, 22 Jan 2015, Heiko Schroeder wrote:


Hi Matthias,


What's problematic with it?
The great problem seems to be cdimage.debian.org.  It is not reachable. 
Even not for netinst-Images.


I have no problems reaching it, so I can't really tell what your problem 
with it is.


You could try downloading them from say:

http://ftp.fi.debian.org/debian-cd/

or

http://ftp.de.debian.org/debian-cd/

Perhaps one of those works better if you have problems with the main 
mirror?


/Mattias Wadenstein


--
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/alpine.deb.2.10.1501221659340.1...@hirohito.acc.umu.se

Re: Jigdo daily images

[Steve McIntyre]

On Fri, Jul 29, 2011 at 10:52:45PM +1000, Adam Baxter wrote:
Hi guys,
A while ago I used daily testing CD-1 ISOs in conjunction with the
hd-media images on a USB stick. The jigdo images I used to build these
from seem to have disappeared, only weekly images are available. Is
this permanent?

Hi Adam,

That must have been really quite a while back. The daily builds have
just been netinst and businesscard sizes for several years... :-)

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
  Getting a SCSI chain working is perfectly simple if you remember that there
  must be exactly three terminations: one on one end of the cable, one on the
  far end, and the goat, terminated over the SCSI chain with a silver-handled
  knife whilst burning *black* candles. --- Anthony DeBoer


-- 
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110729141051.gg28...@einval.com

Re: Jigdo search still broken.

[Simon Paillard]

Hi all,

As we've just got another report about jigdo search, I wonder if there are
updates about moving this service to get it operationnal.

FYI this is tracked by #589477.

On Thu, Oct 07, 2010 at 02:26:30PM +0100, Philip Hands wrote:
 On Wed, 6 Oct 2010 23:33:45 +0200, Richard Atterer atte...@debian.org wrote:
  On Wed, Oct 06, 2010 at 02:29:42PM +0100, Steve McIntyre wrote:
   On Mon, Oct 04, 2010 at 03:47:56PM +0200, thorres wrote:
   the search for packages inside the jigdo files still does not work an
   and leads to a 404 error.
   (http://atterer.org/jigdo/jigdo-search.php?q=foo)
   
   ACK. We should move that content onto a proper Debian service.
   Richard, can you help with that?
  
  Hi - sorry that the search went offline, it happened when I moved servers 
  and I never fixed it. But this should have been on a Debian server in the 
  first place...
  
  The setup consists of a cron job to mirror .jigdo files (which could be 
  omitted if we set this up on cdimage) and a PHP file. Is PHP OK for Debian 
  machines? IIRC perl was preferred...
 
 Hi Richard,
 
 You could run it on free.hands.com (where you already have an account),
 which for the moment has PHP on it.  I now have that on a VM and would
 at some point want to shift the PHP stuff onto a separate VM, so if we
 do that it would be wise to come up with a URL that would survive being
 shifted onto a new machine at some point.  That could of course be done
 by having nginx on free proxy through to the new machine.
 
 If it's more practical to do it elsewhere, that's great, but thought I'd
 mention this option as it seems likely to be the least effort.

-- 
Simon Paillard


-- 
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/2010190135.gr14...@glenfiddich.ikibiki.org

Re: Jigdo search still broken.

[Richard Atterer]

Hello,

On Thu, Nov 11, 2010 at 08:01:35PM +0100, Simon Paillard wrote:
 As we've just got another report about jigdo search, I wonder if there are
 updates about moving this service to get it operationnal.

Hi, it's still on my TODO list, though unfortunately I have been rather 
busy with other things lately.

The thing is also: It's not ideal to put it on my own webspace again, nor 
to use Phil's VM, since neither is an official Debian machine. So either 
way, it would only be a temporary solution. :-|

By the way, thanks for your help with CD vendor entries, Simon - very much 
appreciated! :)

  Richard

-- 
  __   ,
  | ) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/ |  http://atterer.org  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC 8883 
54F7


-- 
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20101112060734.ga21...@arbonne.lan

Re: Jigdo search still broken.

[Philip Hands]

On Wed, 6 Oct 2010 23:33:45 +0200, Richard Atterer atte...@debian.org wrote:
 On Wed, Oct 06, 2010 at 02:29:42PM +0100, Steve McIntyre wrote:
  On Mon, Oct 04, 2010 at 03:47:56PM +0200, thorres wrote:
  Hi there,
  
  the search for packages inside the jigdo files still does not work an
  and leads to a 404 error.
  (http://atterer.org/jigdo/jigdo-search.php?q=foo)
  
  ACK. We should move that content onto a proper Debian service.
  Richard, can you help with that?
 
 Hi - sorry that the search went offline, it happened when I moved servers 
 and I never fixed it. But this should have been on a Debian server in the 
 first place...
 
 The setup consists of a cron job to mirror .jigdo files (which could be 
 omitted if we set this up on cdimage) and a PHP file. Is PHP OK for Debian 
 machines? IIRC perl was preferred...

Hi Richard,

You could run it on free.hands.com (where you already have an account),
which for the moment has PHP on it.  I now have that on a VM and would
at some point want to shift the PHP stuff onto a separate VM, so if we
do that it would be wise to come up with a URL that would survive being
shifted onto a new machine at some point.  That could of course be done
by having nginx on free proxy through to the new machine.

If it's more practical to do it elsewhere, that's great, but thought I'd
mention this option as it seems likely to be the least effort.

Cheers, Phil.

P.S. Alternatively, it looks like you've not logged in since 2003, so if
you don't need the account any more, I could remove it.
-- 
|)|  Philip Hands [+44 (0)20 8530 9560]http://www.hands.com/
|-|  HANDS.COM Ltd.http://www.uk.debian.org/
|(|  10 Onslow Gardens, South Woodford, London  E18 1NE  ENGLAND


pgpx6BQm2IuU5.pgp
Description: PGP signature

Re: Jigdo search still broken.

[Steve McIntyre]

On Mon, Oct 04, 2010 at 03:47:56PM +0200, thorres wrote:
Hi there,

the search for packages inside the jigdo files still does not work an
and leads to a 404 error.
(http://atterer.org/jigdo/jigdo-search.php?q=foo)

ACK. We should move that content onto a proper Debian service.
Richard, can you help with that?

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
We don't need no education.
We don't need no thought control.


-- 
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20101006132942.gd14...@einval.com

Re: Jigdo search still broken.

[Richard Atterer]

On Wed, Oct 06, 2010 at 02:29:42PM +0100, Steve McIntyre wrote:
 On Mon, Oct 04, 2010 at 03:47:56PM +0200, thorres wrote:
 Hi there,
 
 the search for packages inside the jigdo files still does not work an
 and leads to a 404 error.
 (http://atterer.org/jigdo/jigdo-search.php?q=foo)
 
 ACK. We should move that content onto a proper Debian service.
 Richard, can you help with that?

Hi - sorry that the search went offline, it happened when I moved servers 
and I never fixed it. But this should have been on a Debian server in the 
first place...

The setup consists of a cron job to mirror .jigdo files (which could be 
omitted if we set this up on cdimage) and a PHP file. Is PHP OK for Debian 
machines? IIRC perl was preferred...

It doesn't build a word index, so it needs more CPU than needed, but in 
practice this wasn't a problem on the old server.

Cheers,
  Richard

-- 
  __   ,
  | ) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/ |  http://atterer.org  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC 8883 
54F7


-- 
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20101006213344.ga24...@meeep.lan

Re: jigdo files: no source for testing after 3 weeks?

[Steve McIntyre]

Sorry for the delay... :-(

On Sun, Sep 20, 2009 at 01:03:33PM +0100, ael wrote:

 2) The source jigdos for the weekly builds at
http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/
  are missing and have been missing for 3 (or more?) weeks.

 Are the sources built last perhaps, so that any error in any  
 architecture blocks the sources? If so, shouldn't the sources be first?  
 They are really the most important in that one can rebuild and maybe  
 work around build problems for any architecture, but only if they are 
 there.

The cron job that builds the weekly images normally goes in the order:

i386 source amd64 multi powerpc alpha armel hppa ia64 mips mipsel s390 sparc

for reference. What happened here (and has happened a couple of times
in the past) is that a source package moved from one part of the
archive to another (in this case the weirdx package went from contrib
to main) and dak doesn't/didn't handle this case very well. The
debian-cd scripts found that the weirdx .orig.tar.gz file was missing
and bailed out.

I don't always have the time to check up on the build failures, which
is why this happened for 3 weeks running. Once you pointed it out
(thanks!), I spoke to the ftpteam and got them to fix the archive. All
should be fine now AFAICS.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
I suspect most samba developers are already technically insane... Of
 course, since many of them are Australians, you can't tell. -- Linus Torvalds


signature.asc
Description: Digital signature

Re: jigdo only downloads 10 files for my custom install dvd

[Richard Atterer]

Hi Daniel,

if you let jigdo-lite running, it will eventually continue downloading 
further files and copy them to the image ...so it does work!

On Sun, Feb 22, 2009 at 07:29:06PM -0500, Daniel Dadap wrote:
 Does anybody have any ideas why jigdo keeps stopping after downloading 10 
 files?

My intention when I implemented it was that 1) the image file should not be 
fragmented, and 2) the download should terminate early if there is not 
enough room on the hard drive.

For this reason, jigdo-lite writes out the entire image when it first 
creates the .tmp file, filling up missing data with zero bytes. Later runs 
will be much faster because then only the data that was downloaded will be 
copied over into the .tmp file.

Downloading all files first and only then creating the entire image will 
temporarily use up twice as much disk space. With the large size of some 
images, this is not an option IMHO.

(Today, I would probably implement image creation differently though, with 
an explicit check for free disk space, and by using a sparse file in the 
hope that the filing system will take the hint and allocate it 
contiguously. Or you could download in the order of appearance in the 
image file...)

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/¯|  http://atterer.net  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC 8883 
54F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Re: jigdo files isn't found.

[Steve McIntyre]

On Sat, Nov 22, 2008 at 10:35:59AM +0200, Alexander wrote:
   Hi,

 I want to create my new fresh Lenny dvd's, anybody knows what happened
with jigdo weeklies files:

http://cdimage.debian.org/cdimage/weekly-builds/i386/jigdo-dvd/ 

 It is totally absent! I hadn't seen it never before.

Sorry, made a mistake when testing some code changes last night and
deleted the last builds. I'm rebuilding now.

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
When C++ is your hammer, everything looks like a thumb. -- Steven M. Haflich


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Re: jigdo files isn't found.

[Alexander Golovin]

  Thanks you for quick fix it. 
 The all fresh (22-Nov-2008) jigdo files in there again. 
 
 It's ok now ;).
 


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo and BitTorrent (was Secondary CD/DVD Image Downloading)

[Steve Cotton]

On Sun, Aug 31, 2008 at 09:47:39PM -0400, Daniel Dickinson wrote:
 I'm interested.  I know nothing of jigdo, but this would allow me to
 make torrent images available more quickly.  Also, can you do this with
 any pool (such as one from apt-cacher-ng)?

As long as its creates a cache directory (or directory tree) with
.deb files in, Jigdo can use it.  It will also use .debs from
old CDs/DVDs, even if they weren't downloaded with Jigdo.

Details:
file:///usr/share/doc/jigdo-file/debian-jigdo-mini-howto.html#UPDATINGYOURIMAGE
http://atterer.net/jigdo/debian-jigdo-mini-howto/x304.html

The fact that you can use the apt cache is currently buried in the
HOWTO's errata section.



Quick overview of the bittorrent trick:
This isn't my idea, but I can't find where I learned this from to
acknowledge the inventor.

Once Jigdo has finished finding .debs on your PC, it would
normally complete the ISO by downloading the missing files from a
mirror.  At this point press ctrl-C, and it will leave an .iso.tmp
file which is the DVD (and a valid filesystem), except that the
missing .debs appear as files of the right size with their content
filled replaced by zeros.

Move the .iso.tmp to the location that BitTorrent / BitTornado
would download it to (with default BT settings, just rename it to
.iso).  BT will check the existing data for corruption, using the
checksums in the .torrent file.  Any block that was completely
filled in by Jigdo will check correctly; the rest will be marked
as corrupt and downloaded again via bittorrent.

(TODO: check the behaviour of other bittorrent implementations,
and write this in a more accessible way.)

Steve


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo in SoC [Re: What CDs and DVDs should we produce for lenny?]

[Rick Thomas]

On Mar 17, 2008, at 7:23 PM, Richard Atterer wrote:

Re: What CDs/DVDs? I agree with many others here: Produce images  
(and
.jigdos) for all images, but only mirror a limited subset via HTTP/ 
FTP.


The commercial CD/DVD sellers can produce those CDs and DVDs that  
aren't available directly.  That's part of their value added.


Another part of their value added is bandwidth savings.  If I'm  
working in the highlands of Papua New Guinea, on a 600-MB/month  
bandwidth budget, I'll be happy to pay somebody else (in, say,  
Australia or in one of the PNG coastal towns) to do the downloading  
and compilation of a full set of installation CDs.


Giving the commercial resellers the tools (such as jigdo) to do this  
is an important part of Debian's social responsibility.  Giving them  
tools (such as debian-cd) to customize and extend their offerings is  
part of the open source philosophy.



Rick


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo Lenny weekly builds missing 12 files in DVD1

[Jasbir Singh Dhillon]

Thanks Steve

Just wanted someone to know that something was wrong. The 25 Feb images are OK 
so far. Have got the DVD2 download so far. 

Thanks and best regards

Capt Jasbir Singh Dhillon


On Thursday 28 February 2008 03:16, Steve McIntyre wrote:
 On Sat, Feb 23, 2008 at 07:43:33AM +0530, Jasbir Singh Dhillon wrote:
 For each file jigdo reported.
 
 ERROR 404: Not Found.
 
 Last weeks Jigdo downloads had 12 Files missing in DVD1
 
 Jigdo advises to use rsync but the rsync servers are available for only
  the stable branch. If any one knows of any servers offering the weekly
  builds with rsync I would be gratefull for the URL.

 Apologies for the missing files, there was a config mistake on the
 snapshot server that meant it was no longer automatically taking
 snapshots of the weekly builds. I've fixed that now, so for the builds
 dated from Monday 25th things should be fine. Unfortunately, some of
 the older files for the older weeklies are going to be missing
 altogether. Sorry... :-(


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Re: Jigdo Lenny weekly builds missing 12 files in DVD1

[Steve McIntyre]

On Sat, Feb 23, 2008 at 07:43:33AM +0530, Jasbir Singh Dhillon wrote:

For each file jigdo reported.

ERROR 404: Not Found.

Last weeks Jigdo downloads had 12 Files missing in DVD1

Jigdo advises to use rsync but the rsync servers are available for only the 
stable branch. If any one knows of any servers offering the weekly builds 
with rsync I would be gratefull for the URL.

Apologies for the missing files, there was a config mistake on the
snapshot server that meant it was no longer automatically taking
snapshots of the weekly builds. I've fixed that now, so for the builds
dated from Monday 25th things should be fine. Unfortunately, some of
the older files for the older weeklies are going to be missing
altogether. Sorry... :-(

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
Into the distance, a ribbon of black
Stretched to the point of no turning back


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Re: Jigdo Lenny weekly builds missing 12 files in DVD1

[Jasbir Singh Dhillon]

Now I have tried to download the 21 Feb Lenny Jigdo Image DVD1 reusing the 
Image I had from last week and again found files missing ( 10 files missing). 
Last weeks Jigdo downloads had 12 Files missing in DVD1. These were not 
available on the snapshot, which jigdo tries automatically

This is the list of files missing
 
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/t/ttf-indic-fonts/ttf-gujarati-fonts_0.5.1_all.deb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/t/tasksel/tasksel_2.73_all.deb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/d/debian-installer-utils/di-utils-mapdevfs_1.55_i386.udeb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/t/ttf-indic-fonts/ttf-tamil-fonts_0.5.1_all.deb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/t/ttf-indic-fonts/ttf-punjabi-fonts_0.5.1_all.deb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/t/ttf-indic-fonts/ttf-devanagari-fonts_0.5.1_all.deb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/t/ttf-indic-fonts/ttf-malayalam-fonts_0.5.1_all.deb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/t/ttf-indic-fonts/ttf-bengali-fonts_0.5.1_all.deb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/libn/libnet-dns-perl/libnet-dns-perl_0.63-1_i386.deb
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/i/installation-report/installation-report_2.32_all.deb

For each file jigdo reported.

ERROR 404: Not Found.

Last weeks Jigdo downloads had 12 Files missing in DVD1

Jigdo advises to use rsync but the rsync servers are available for only the 
stable branch. If any one knows of any servers offering the weekly builds 
with rsync I would be gratefull for the URL.

Jasbir Singh Dhillon


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo Lenny weekly builds missing 12 files in DVD1

[Jasbir Singh Dhillon]

References: [EMAIL PROTECTED]


Thanks for the reply. I am aware of the possibility of loop mounting the image 
downloaded so far and using that with next weeks jigdo. 

I also wrote that I cannot find the mirrror offering the Lenny DVD images with 
rsync.
The only rsync servers I found were for the stable version Etch.

Here is the rsync output for the server offering Lenny weekly images

[EMAIL PROTECTED]:~$ rsync http://cdimage.debian.org/cdimage/weekly-builds/
ssh: http: Name or service not known
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: unexplained error (code 255) at io.c(453) [receiver=2.6.9]

Here is what you get when you querry a rsync server. the first few lines are 
removed

[EMAIL PROTECTED]:~$ rsync debian.inode.at::debian/debian-cd/
--- (var lines removed)
---
drwxr-xr-x4096 2008/02/19 04:52:31 .
lrwxrwxrwx   6 2008/02/19 04:52:31 current
drwxr-xr-x4096 2008/02/19 04:52:31 4.0_r3
drwxr-xr-x4096 2005/05/23 22:20:12 project
[EMAIL PROTECTED]:~$  

You see no Lenny images   

If you can point me in the right direction for a mirror offering the Lenny 
image with rsync, then maybe that will help.

Else will continue downloading the other DVD's till I see the updated Jigdo 
files.

Here in India with the connection plan I have we are getting download speeds 
of about 30 KB/s so one DVD will take a couple of days. 

Thanks again
Jasbir Singh Dhillon



Re: Jigdo Lenny weekly builds missing 12 files in DVD1
To: Jasbir Singh Dhillon [EMAIL PROTECTED]
Cc: debian-cd@lists.debian.org
Subject: Re: Jigdo Lenny weekly builds missing 12 files in DVD1
From: Don Wright [EMAIL PROTECTED]
Date: Mon, 18 Feb 2008 22:18:45 -0600
Message-id: [EMAIL PROTECTED]
In-reply-to: [EMAIL PROTECTED]
References: [EMAIL PROTECTED]
On Tue, 19 Feb 2008 06:57:54 +0530, Jasbir Singh Dhillon
[EMAIL PROTECTED] wrote:

I was trying to download the weekly lenny DVD's using the Jigdo files dated 
11 
Feb 08. 12 Files are missing.

If you're quick, you can rename the temporary .iso file created by jigdo
as if it succeeded and use rsync to copy only the missing pieces from
the full DVD image. (Once the new week's builds arrive, this trick won't
work.) You could do the same with bittorrent if there were DVD torrents
available.

If you miss the deadline, you can still use the damaged .iso as a seed
for jigdo to create the newer DVD or CD images.

Most people find that only the first couple of CDs are needed for an
offline Debian install. Any additional packages can be downloaded by
your new Debian system after the installer completes.
  --Don


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo Lenny weekly builds missing 12 files in DVD1

[Don Wright]

On Tue, 19 Feb 2008 06:57:54 +0530, Jasbir Singh Dhillon
[EMAIL PROTECTED] wrote:

I was trying to download the weekly lenny DVD's using the Jigdo files dated 11 
Feb 08. 12 Files are missing.

If you're quick, you can rename the temporary .iso file created by jigdo
as if it succeeded and use rsync to copy only the missing pieces from
the full DVD image. (Once the new week's builds arrive, this trick won't
work.) You could do the same with bittorrent if there were DVD torrents
available.

If you miss the deadline, you can still use the damaged .iso as a seed
for jigdo to create the newer DVD or CD images.

Most people find that only the first couple of CDs are needed for an
offline Debian install. Any additional packages can be downloaded by
your new Debian system after the installer completes.
  --Don

Re: Re: Jigdo for Debian 2.2 (potato) for PPC

[Rick Thomas]

Brian wrote:

Thanks much for the suggestion about the alternate kernel for  
Sarge.  That's a terrific idea and one which I never would have  
thought to explore without your help!



Hi Brian,

If you haven't already gotten your emulated machine running Sarge,  
I've been told that the recently announced Debian Sarge release 3.1r7  
will have iso's available at http://cdimage.debian.org/cdimage/ 
archive/ in a couple of days.


Unfortunately, the 3.1r6 images (available there now) were created  
before Sarge changed from stable to oldstable.  So they don't  
offer to actually install Sarge (it should be called oldstable if  
they offered it, but all they have is stable, testing, and  
unstable)  This bug is supposedly fixed in 3.1r7.


Good luck, and have fun with Debian Sarge!

Rick


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Re: Jigdo for Debian 2.2 (potato) for PPC

[Steve McIntyre]

On Mon, Jan 07, 2008 at 05:04:38AM -0500, Rick Thomas wrote:
Brian wrote:

Thanks much for the suggestion about the alternate kernel for  
Sarge.  That's a terrific idea and one which I never would have  
thought to explore without your help!


Hi Brian,

If you haven't already gotten your emulated machine running Sarge,  
I've been told that the recently announced Debian Sarge release 3.1r7  
will have iso's available at http://cdimage.debian.org/cdimage/ 
archive/ in a couple of days.

Unfortunately, the 3.1r6 images (available there now) were created  
before Sarge changed from stable to oldstable.  So they don't  
offer to actually install Sarge (it should be called oldstable if  
they offered it, but all they have is stable, testing, and  
unstable)  This bug is supposedly fixed in 3.1r7.

Hi guys,

Unfortunately there is another major issue with 3.1r7 that we've found
in last-minute testing just before I released them. I'm hoping to get
a fix for that shortly, but for now the 3.1r7 images are on hold and
not useable. :-(

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
Who needs computer imagery when you've got Brian Blessed?


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo Debian 4.0r2

[Steve McIntyre]

On Sun, Jan 06, 2008 at 08:24:31PM +0100, S. Pfeffer wrote:
Hello,

sorry if this problem has been already posted here. I just subscribed
the list.
There ist a Problem with the jido files for the i386 and amd64
architecture. In the jigdo file ist the path
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-i386/20070308etch1/images/
which does not exist. The path is
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308/images/
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-i386/20070308/images/
Ist there a workaround, or will a symlink on the server be created?

Hi,

Apologies - a simple mistake on the snapshot server. I'm updating that
snapshot now so it will be ready when needed. In the meantine, you
should not be needing the snapshot at all unless your local mirror is
out of date or incomplete. I'd suggest you check on that.

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
Who needs computer imagery when you've got Brian Blessed?


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo Debian 4.0r2

[S. Pfeffer]

Am 07.01.2008 18:35, Steve McIntyre schrieb:

 Hi,
 
 Apologies - a simple mistake on the snapshot server. I'm updating that
 snapshot now so it will be ready when needed. In the meantine, you
 should not be needing the snapshot at all unless your local mirror is
 out of date or incomplete. I'd suggest you check on that.
 

Thank you for the quick fixing


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Re: jigdo Debian 4.0r2

[Alexander Golovin]

--20:18:00--
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/initrd.gz
   =
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/initrd.gz'
Resolving us.cdimage.debian.org... 83.142.228.128
Connecting to us.cdimage.debian.org|83.142.228.128|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3,812,181 (3.6M) [application/x-gzip]

100%[==]
3,812,181486.31K/sETA 00:00

20:18:09 (473.33 KB/s) -
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/initrd.gz'
 saved [3812181/3812181]

--20:18:09--
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/vmlinuz
   =
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/vmlinuz'
Reusing existing connection to us.cdimage.debian.org:80.
HTTP request sent, awaiting response... 200 OK
Length: 1,514,347 (1.4M) [text/plain]

100%[==]
1,514,347432.03K/sETA 00:00

20:18:14 (339.44 KB/s) -
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/vmlinuz'
 saved [1514347/1514347]

--20:18:14--
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/initrd.gz
   =
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/initrd.gz'
Reusing existing connection to us.cdimage.debian.org:80.
HTTP request sent, awaiting response... 200 OK
Length: 10,502,827 (10M) [application/x-gzip]

100%[==]
10,502,827   485.66K/sETA 00:00

20:18:36 (456.27 KB/s) -
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/initrd.gz'
 saved [10502827/10502827]


FINISHED --20:18:36--
Downloaded: 15,829,355 bytes in 3 files
Found 3 of the 3 files required by the template
Successfully created `debian-40r2-amd64-DVD-1.iso'


 I have updated my mirror, everything is all right. 
Thanks.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo Debian 4.0r2

[Alexander Golovin]

Hi, 

I have the same problem:


Merging parts from `file:' URIs, if any...
Found 0 of the 3 files required by the template
Copied input files to temporary file `debian-40r2-amd64-DVD-1.iso.tmp' -
repeat command and supply more files to continue
--22:59:13--
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/initrd.gz
   =
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/initrd.gz'
Resolving us.cdimage.debian.org... 83.142.228.128
Connecting to us.cdimage.debian.org|83.142.228.128|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
22:59:14 ERROR 404: Not Found.

--22:59:14--
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/vmlinuz
   =
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/vmlinuz'
Reusing existing connection to us.cdimage.debian.org:80.
HTTP request sent, awaiting response... 404 Not Found
22:59:14 ERROR 404: Not Found.

--22:59:14--
http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/initrd.gz
   =
`debian-40r2-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/cdrom/gtk/initrd.gz'
Reusing existing connection to us.cdimage.debian.org:80.
HTTP request sent, awaiting response... 404 Not Found
22:59:14 ERROR 404: Not Found.


FINISHED --22:59:14--
Downloaded: 0 bytes in 0 files
Skipping object `debian-40r2-amd64-DVD-1.iso.tmpdir' (No such file or
directory)
Found 0 of the 3 files required by the template
Copied input files to temporary file `debian-40r2-amd64-DVD-1.iso.tmp' -
repeat command and supply more files to continue

-
Aaargh - 3 files could not be downloaded. This should not
happen! Depending on the problem, it may help to retry downloading
the missing files.
Also, you could try changing to another Debian or Non-US server,
in case the one you used is out of sync.

However, if all the files downloaded without errors and you
still get this message, it means that the files changed on the
server, so the image cannot be generated.
As a last resort, you could try to complete the CD image download
by fetching the remaining data with rsync.

Press Return to retry downloading the missing files.
Press Ctrl-C to abort. (If you re-run jigdo-lite later, it will
resume from here, the downloaded data is not lost if you press
Ctrl-C now.)



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo Debian 4.0r2

[Carlos Carvalho]

S. Pfeffer ([EMAIL PROTECTED]) wrote on 6 January 2008 20:24:
 sorry if this problem has been already posted here. I just subscribed
 the list.
 There ist a Problem with the jido files for the i386 and amd64
 architecture. In the jigdo file ist the path
 http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/
 http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-i386/20070308etch1/images/
 which does not exist. The path is
 http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308/images/
 http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-i386/20070308/images/
 Ist there a workaround, or will a symlink on the server be created?

You mention 4.0_r2 in the subject and snapshot in the paths. I don't
understand how a stable release would mention snapshot...

I generated all the images from the jigdo files without problem.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Re: jigdo Debian 4.0r2

[Alexander Golovin]

On Sun, 2008-01-06 at 19:25 -0200, Carlos Carvalho wrote: 
 S. Pfeffer ([EMAIL PROTECTED]) wrote on 6 January 2008 20:24:
  sorry if this problem has been already posted here. I just subscribed
  the list.
  There ist a Problem with the jido files for the i386 and amd64
  architecture. In the jigdo file ist the path
  
 http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/
  
 http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-i386/20070308etch1/images/
  which does not exist. The path is
  
 http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308/images/
  
 http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-i386/20070308/images/
  Ist there a workaround, or will a symlink on the server be created?
 
 You mention 4.0_r2 in the subject and snapshot in the paths. I don't
 understand how a stable release would mention snapshot...
 
 I generated all the images from the jigdo files without problem.
 

 The problem of creation of images remains is opened for me. Please
share successful experience of creation of images, in more detail.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Re: jigdo Debian 4.0r2

[Carlos Carvalho]

Alexander Golovin ([EMAIL PROTECTED]) wrote on 7 January 2008 00:07:
 On Sun, 2008-01-06 at 19:25 -0200, Carlos Carvalho wrote: 
  S. Pfeffer ([EMAIL PROTECTED]) wrote on 6 January 2008 20:24:
   sorry if this problem has been already posted here. I just subscribed
   the list.
   There ist a Problem with the jido files for the i386 and amd64
   architecture. In the jigdo file ist the path
   
  http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308etch1/images/
   
  http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-i386/20070308etch1/images/
   which does not exist. The path is
   
  http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-amd64/20070308/images/
   
  http://us.cdimage.debian.org/cdimage/snapshot/Debian/dists/etch/main/installer-i386/20070308/images/
   Ist there a workaround, or will a symlink on the server be created?
  
  You mention 4.0_r2 in the subject and snapshot in the paths. I don't
  understand how a stable release would mention snapshot...
  
  I generated all the images from the jigdo files without problem.
  
 
  The problem of creation of images remains is opened for me. Please
 share successful experience of creation of images, in more detail.

I just downloaded the jigdo files from cdimage.debian.org/debian-cd/
and ran the script that creates the isos. All packages were taken from
our mirror, which is complete and closely follows ftp-master. There
were no missing packages or checksum problems. I'm sure about this
because I set maxMissing=0.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo for Debian 2.2 (potato) for PPC

[Richard Atterer]

Hi Brian,

On Sat, Dec 08, 2007 at 12:49:18PM -0700, Brian Fisk wrote:
 I am trying to see if I can get Debian to run on a Power PC emulator
 (PearPC).  I know that the emulator does not support 2.6 kernels, so I must
 try to find a version with kernel 2.4, which is to say I want Debian
 2.2potato or eariler.

Phew, that's pretty tough...

Some pointers you may not be aware of:
http://archive.debian.org/dists/ contains the _packages_ of all old 
releases, but no CD images.
http://cdimage.debian.org/cdimage/archive/ contains CD images starting 
with 3.1r0 (sarge).

I remember that potato was the first version for which I created .jigdo 
files, but I don't have them anymore! :-/

One possible approach: 
http://www.debian.org/releases/sarge/powerpc/release-notes/ch-installing.en.html
 
seems to suggest that while a 2.6 kernel is used by default on PowerPC for 
Debian 3.1, a 2.4 kernel may also be available. There are no details, but I 
guess entering some special value at the installer prompt will boot a 2.4 
kernel.

Maybe ask on [EMAIL PROTECTED], some old-time users there may be able to 
supply old ISO images, or suggestions on how to get your (or some other) 
emulator to work...

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/¯|  http://atterer.net  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC 8883 
54F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo for Debian 2.2 (potato) for PPC

[Brian Fisk]

Hi Richard,

Thanks much for the suggestion about the alternate kernel for Sarge.  That's
a terrific idea and one which I never would have thought to explore without
your help!

Best wishes,

Brian

On Dec 8, 2007 1:38 PM, Richard Atterer [EMAIL PROTECTED] wrote:

 Hi Brian,

 On Sat, Dec 08, 2007 at 12:49:18PM -0700, Brian Fisk wrote:
  I am trying to see if I can get Debian to run on a Power PC emulator
  (PearPC).  I know that the emulator does not support 2.6 kernels, so I
 must
  try to find a version with kernel 2.4, which is to say I want Debian
  2.2potato or eariler.

 Phew, that's pretty tough...

 Some pointers you may not be aware of:
 http://archive.debian.org/dists/ contains the _packages_ of all old
 releases, but no CD images.
 http://cdimage.debian.org/cdimage/archive/ contains CD images starting
 with 3.1r0 (sarge).

 I remember that potato was the first version for which I created .jigdo
 files, but I don't have them anymore! :-/

 One possible approach:
 
 http://www.debian.org/releases/sarge/powerpc/release-notes/ch-installing.en.html
 
 seems to suggest that while a 2.6 kernel is used by default on PowerPC for
 Debian 3.1, a 2.4 kernel may also be available. There are no details, but
 I
 guess entering some special value at the installer prompt will boot a 2.4
 kernel.

 Maybe ask on [EMAIL PROTECTED], some old-time users there may be able
 to
 supply old ISO images, or suggestions on how to get your (or some other)
 emulator to work...

 Cheers,

  Richard

 --
  __   _
  |_) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/¯|  http://atterer.net  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC
 8883 54F7
  ¯ '` ¯

Re: Jigdo testing DVD snapshots outdated?

[Steve McIntyre]

On Wed, Jul 04, 2007 at 07:27:38AM +0200, Maarten van Geijn wrote:
Dear Debian CD team,

The jigdo CD-images, which are available at 
http://cdimage.debian.org/cdimage/weekly-builds/i386/jigdo-dvd/
seem to be outdated; they are dated at May 28th, 2007.

Do you know if there are any technical problems the cdimage project 
suffers from? And when are the snapshots expedted to be up-to-date again?
Is there any way I can support the Debian team to help fixing this problem?

Hi Maarten,

At the moment there isn't much point in producing the weekly testing
images, as we know they would be broken. Testing should hopefully sync
to a useful enough state for a working build soon.

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
I suspect most samba developers are already technically insane... Of
 course, since many of them are Australians, you can't tell. -- Linus Torvalds


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo (was Re: too many CD ISOs)

[Richard Atterer]

Hi, sorry for the late reply...

On Thu, Apr 26, 2007 at 01:44:05PM +0100, Steve McIntyre wrote:
 There's also a couple more changes/updates that I'd like to make
 in/around jigdo soon-ish:
 
  * Move over to using bzip2 rather than gzip for our template files.
That should be simple enough now that all the clients in stable
will support bzip2, I assume.

Yes, it should work AFAICT!

  * Start using sha1/256 internally as well as/instead of md5sum, or at
least for the whole-image checksums. Md5sum is looking weak these
days.

I thought about SHA1 when I first came up with the file format. At the 
time, MD5 was already considered weakend because different content with 
matching MD5s had been found. Still, in the end I decided against SHA1 
because the individual file checksums are not security sensitive - CRC64 
would be just as fine IMHO.

However, in hindsight I probably should have used SHA1 for the template 
file hash and for the hash of the final image.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/¯|  http://atterer.net  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC 8883 
54F7
  ¯ '` ¯


signature.asc
Description: Digital signature

Re: jigdo for olpc

[Mark Eichin]

Richard Atterer [EMAIL PROTECTED] writes:


 Finally, a simple solution: Mount each image just once and copy the 
 contents out of the loop mount. If there's not enough space on the server, 
 use hard links for files which are shared between different versions of the 
 images - I guess relatively few files will change between releases?

If you're just copying particular files, rather than cloning the whole
disk, isoinfo -x is one way to avoid the need for loopback
altogether...


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo for olpc

[Richard Atterer]

Hi Carl,

On Thu, Mar 01, 2007 at 11:11:36PM -0600, Carl Karsten wrote:
 1. Will jigdo work with other fs images, like the ext3 ones?

Yes, but :-)
It will work as long as you take care that the files inside the filesystem 
are not fragmented. jigdo will only work if the data of a file is a 
contiguous region of the filesystem. Usually, you won't run into trouble if 
you create a fresh FS and copy data into it.

 2. If many versions (100) of the the squishfs iso is stored on the server, 
 and it takes 3 mount -o loop's to get to the actual files:
[...]
 Will the server have a problem with the 300 loopbacks?  If so, is there 
 some way to make apache bring them up and down automatically?

300 is a large number. You'll need to give the right parameter to the loop 
module to be able to do this. Not sure whether it'll work, but I'd give it 
a try, I can't see a reason why it shouldn't work.
(You might be able to eliminate the .iso loop mount, by specifying the 
offset of the compressed filesystem.)

Otherwise, you can write a PHP file which serves the content manually 
after mounting the images. This doesn't sound very nice to me - you might 
still run into trouble if too many files are requested simultaneously.

Finally, a simple solution: Mount each image just once and copy the 
contents out of the loop mount. If there's not enough space on the server, 
use hard links for files which are shared between different versions of the 
images - I guess relatively few files will change between releases?

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/¯|  http://atterer.net  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC 8883 
54F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo list missing 3.1r4

[Richard Atterer]

On Sat, Feb 24, 2007 at 05:18:42AM +0100, Matt Kraai wrote:
 On Sat, Feb 24, 2007 at 03:52:21PM +1300, Austin Green wrote:
  The page List of Debian .jigdo files at
  http://atterer.net/jigdo/jigdo-search.php?list
  does not include the latest stable point release (3.1r4)
 
 I think someone from the debian-cd team will need to address this.

I have manually restarted the cron job on atterer.net, and it's downloading 
the new .jigdo files as I write this. ATM, I can't figure out why it has 
stopped mirroring from cdimage.d.o, everything seems to work OK. :-/

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key: 888354F7
  | \/¯|  http://atterer.net  |  08A9 7B7D 3D13 3EF2 3D25  D157 79E6 F6DC 8883 
54F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite download of DVD #3 of AMD64 RC1 fails due to missing file...

[Steve McIntyre]

On Mon, Jan 29, 2007 at 11:24:38AM +, Michael Fothergill wrote:
DearDebianists,

A similar error to the one I reported the other day concerning a jigdo 
download of a DVD from the amd64 RC1 Etch release.

A missing file has interrupted the download again.

I am ccing the debian-cd list on this as requested by Steve Mcintyre who 
kindly fixed the problem the last time it occured.

This time it is DVD#3 of the amd64 set.

The problem file is: aladin_1.19-7_amd64.deb.

snip

Fixed, I hope. Same problem as last time; I'll be much happier once we
get Etch released and then there won't be the same confusion between
the old unofficial amd64 packages and new official ones with exactly
the same name... :-/

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
When C++ is your hammer, everything looks like a thumb. -- Steven M. Haflich


signature.asc
Description: Digital signature

Re: jigdo-lite download of DVD #3 of AMD64 RC1 fails due to missing file...

[Michael Fothergill]

Michael Fothergill wrote:

DearDebianists,

A similar error to the one I reported the other day concerning a jigdo
download of a DVD from the amd64 RC1 Etch release.

A missing file has interrupted the download again.

I am ccing the debian-cd list on this as requested by Steve Mcintyre who
kindly fixed the problem the last time it occured.

This time it is DVD#3 of the amd64 set.

The problem file is: aladin_1.19-7_amd64.deb.

snip

Fixed, I hope.


Yes it is fixed.  I made a complete DVD image with no errors.  I now have 
copies of all three images on DVDs that have no errors according to the 
installer integrity check.
When Etch is released as stable I will re do the DVD set I have for the 
AMD-64 and both the DVD and CD sets for the i386 boxes I use.  I will use 
the RC1 images I have to bootstrap the jigdo process of making the new 
images.  If I find any errors I will post them up on the site as before.


Regards

Michael Fothergill

Same problem as last time; I'll be much happier once we

get Etch released and then there won't be the same confusion between
the old unofficial amd64 packages and new official ones with exactly
the same name... :-/

--
Steve McIntyre, Cambridge, UK.
[EMAIL PROTECTED]
When C++ is your hammer, everything looks like a thumb. -- Steven M. 
Haflich




 signature.asc 


_
MSN Hotmail is evolving – check out the new Windows Live Mail 
http://ideas.live.com



--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo files for 1.4 GB (8cm) DVDs?

[W. Borgert]

Steve McIntyre [EMAIL PROTECTED] wrote:
 True. Basically, we can provide installation images for whatever media
 sizes people want to use, but the more we make the longer it will take
 and the more confusion it will cause for inexperienced people looking
 to download the official images.

8cm DVDs are not very common, but as I wrote, it's a very
handy format.  E.g. it fits into shirt pockets :-) I don't
think, that people become confused, if there is an 8cm
subdirectory with some jigdo files.  Maybe it would be OK
to provide for one 8cm DVD (contents of the first two CDs,
about 1200 packages).  This would make a nice little thing
as promotional gift at booths etc.

Cheers, WB


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo files for 1.4 GB (8cm) DVDs?

[Alexander Schmehl]

* W. Borgert [EMAIL PROTECTED] [050103 17:10]:

 can Debian provide for jigdo files for 1.4 GB (8cm) DVDs?
 That would be a nice competitive differentiation, as no
 other distro - AFAIK - is available in that handy format.

Does this serve any other purpose than they don't have it?  Playing
disc jockey with ~8 small DVDs, if 2 would be sufficient doesn't sound
very usefull for me.  Are there any DVD drives, which can only use 8cm
discs?  Didn't know of any.  Nor did I saw 8cm blank DVDs.


Yours sincerely,
  Alexander


signature.asc
Description: Digital signature

Re: Jigdo files for 1.4 GB (8cm) DVDs?

[Steve McIntyre]

On Mon, Jan 03, 2005 at 05:43:56PM +0100, Alexander Schmehl wrote:
* W. Borgert [EMAIL PROTECTED] [050103 17:10]:

 can Debian provide for jigdo files for 1.4 GB (8cm) DVDs?
 That would be a nice competitive differentiation, as no
 other distro - AFAIK - is available in that handy format.

Does this serve any other purpose than they don't have it?  Playing
disc jockey with ~8 small DVDs, if 2 would be sufficient doesn't sound
very usefull for me.  Are there any DVD drives, which can only use 8cm
discs?  Didn't know of any.  Nor did I saw 8cm blank DVDs.

True. Basically, we can provide installation images for whatever media
sizes people want to use, but the more we make the longer it will take
and the more confusion it will cause for inexperienced people looking
to download the official images.

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
Is there anybody out there?

Re: jigdo-file problems creating woody r4 images

[Richard Atterer]

On Mon, Jan 03, 2005 at 05:32:58PM +, Steve McIntyre wrote:
 Several of the jigdo files I'm creating end up using the following file:
 
 LV13Ru06DGf3YL3ExzFEfw=Debian:pool/main/l/lablgl/liblablgl-ocaml_1.01-2_mips.deb

That checksum is incorrect! :-/ This .deb has a checksum of 
QYrvxbZD6hsrbVCjL1sAWQ on all the mirrors I've checked.

The reason for that problem is probably an inconsistent jigdo cache file. Run

  jigdo-file --cache ~/jigdo-cache.db md5 
/mymirror/pool/main/l/lablgl/liblablgl-ocaml_1.01-2_mips.deb

to verify this. If the wrong sum (LV...) is printed, touch the file and 
repeat the command.

Of course, the better safe than sorry approach would be to delete 
~/jigdo-cache.db, but it'll take a while to rebuild... :-/

I've downloaded your template file and at first sight jigdo-file ls -t
woody-powerpc-1.template shows some _really_ weird things: Our LV checksum 
appears in numerous places all over the image.

[...some intelligent guesses later:]
That LV checksum is the md5 checksum for 53818 bytes of zeroes. :-((

That explains a few things - wherever there are longer stretches of zeroes 
in the image, references to the zero file are repeated many times.

More importantly, if such an entry made its way into the jigdo cache, this 
means that jigdo-file will slow to an absolute crawl for any images which 
contain larger stretches of zeroes - such as the DVD images!! :-( This 
might explain why you haven't been quite that content with jigdo-file's 
performance of late!

So how did the all-0 entry end up in jigdo-file's cache? I have no idea, 
and this looks like one of those horrible bugs which are impossible to 
find. :-/

One possible reason which would allow me to blame someone else ;-) would
be that jigdo-file scanned this file for the first time when it was just
being mirrored. Does rsync (or whatever mirror method you use) first set
the extent and timestamp of a file and then transfer the data? Hmm, that 
sounds unlikely - any suggestions on how I could track this down?

 I've used JTE to generate the other jigdo files, and it doesn't
 exhibit this problem thankfully...

If you use it for the release images, this will also be a nice way of 
testing it and ironing out any remaining bugs - IMHO, this is good since 
you said you want to use it for the sarge release images!

I just wish I could find the cause for this problem with jigdo-file. :-/

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-file problems creating woody r4 images

[Steve McIntyre]

On Mon, Jan 03, 2005 at 08:50:28PM +0100, Richard Atterer wrote:
On Mon, Jan 03, 2005 at 05:32:58PM +, Steve McIntyre wrote:
 Several of the jigdo files I'm creating end up using the following file:
 
 LV13Ru06DGf3YL3ExzFEfw=Debian:pool/main/l/lablgl/liblablgl-ocaml_1.01-2_mips.deb

That checksum is incorrect! :-/ This .deb has a checksum of 
QYrvxbZD6hsrbVCjL1sAWQ on all the mirrors I've checked.

The reason for that problem is probably an inconsistent jigdo cache file. Run

  jigdo-file --cache ~/jigdo-cache.db md5 
 /mymirror/pool/main/l/lablgl/liblablgl-ocaml_1.01-2_mips.deb

to verify this. If the wrong sum (LV...) is printed, touch the file and 
repeat the command.

Of course, the better safe than sorry approach would be to delete 
~/jigdo-cache.db, but it'll take a while to rebuild... :-/

My mirror _and_ the jigdo-cache.db both give the right answer
(QYrvxbZD6hsrbVCjL1sAWQ), which is the really confusing part.

I've downloaded your template file and at first sight jigdo-file ls -t
woody-powerpc-1.template shows some _really_ weird things: Our LV checksum 
appears in numerous places all over the image.

[...some intelligent guesses later:]
That LV checksum is the md5 checksum for 53818 bytes of zeroes. :-((

OK, that would explain _something_.

That explains a few things - wherever there are longer stretches of zeroes 
in the image, references to the zero file are repeated many times.

More importantly, if such an entry made its way into the jigdo cache, this 
means that jigdo-file will slow to an absolute crawl for any images which 
contain larger stretches of zeroes - such as the DVD images!! :-( This 
might explain why you haven't been quite that content with jigdo-file's 
performance of late!

So how did the all-0 entry end up in jigdo-file's cache? I have no idea, 
and this looks like one of those horrible bugs which are impossible to 
find. :-/

Quite.

One possible reason which would allow me to blame someone else ;-) would
be that jigdo-file scanned this file for the first time when it was just
being mirrored. Does rsync (or whatever mirror method you use) first set
the extent and timestamp of a file and then transfer the data? Hmm, that 
sounds unlikely - any suggestions on how I could track this down?

No, rsync is quite clever. It grabs new files down as temporary file
names and then renames them into place when they're finished.

 I've used JTE to generate the other jigdo files, and it doesn't
 exhibit this problem thankfully...

If you use it for the release images, this will also be a nice way of 
testing it and ironing out any remaining bugs - IMHO, this is good since 
you said you want to use it for the sarge release images!

It's being used now by Manty. There's a small issue with m68k at the
moment, but otherwise it seems to work just fine. There are problems
using it with woody for some of the architectures where the boot stuff
has changed for sarge and re-writing it for a couple of woody point
releases is just too much effort. Hence I'm using the original scripts
and jigdo-file for the awkward arches (mipsel, ppc and sparc for now).

I just wish I could find the cause for this problem with jigdo-file. :-/

If it helps as a clue, this file is _exactly_ the same one that caused
me pain for the woody r3 images. That's why I just commented it out
from the jigdofilelist for the next run. Once I have this run
finished, I can re-run with whatever debug enabled that you'd
like. Hell, if it helps I can give you access to the system here to
play with it yourself.

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
I've only once written 'SQL is my bitch' in a comment. But that code 
 is in use on a military site... -- Simon Booth


signature.asc
Description: Digital signature

Re: jigdo-file problems creating woody r4 images

[Richard Atterer]

On Mon, Jan 03, 2005 at 08:12:40PM +, Steve McIntyre wrote:
 My mirror _and_ the jigdo-cache.db both give the right answer
 (QYrvxbZD6hsrbVCjL1sAWQ), which is the really confusing part.

Ah - well, at least this makes it clear that it's a bug in jigdo-file 
itself, and not a cache problem.

 If it helps as a clue, this file is _exactly_ the same one that caused me
 pain for the woody r3 images. That's why I just commented it out from the
 jigdofilelist for the next run. Once I have this run finished, I can
 re-run with whatever debug enabled that you'd like. Hell, if it helps I
 can give you access to the system here to play with it yourself.

So it's reproducible? That's better than nothing... if you can find the
time once the r4 images are out, please rerun the jigdo-file command with
the switches --report=noprogress --debug=make-template - many thanks!

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite --noask --scan PATH :IT works!

[Richard Atterer]

On Tue, Dec 07, 2004 at 07:39:08AM +0700, Rahmat M. Samik-Ibrahim wrote:
 BTW, I am just wondering how jigdo-file-cache.db works?

Maybe have a look at 
http://atterer.net/jigdo/jigdo-file.html#CACHE-FILES, which has some 
information.

 Shall we delete it once in a while?

That shouldn't be necessary, as stale entries in the cache are expired
after a while.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite --noask --scan PATH :IT works!

[Robert Parker]

On Tuesday 07 December 2004 11:39, Rahmat M. Samik-Ibrahim wrote:
 On Tue, 2004-11-30 at 19:40, Richard Atterer wrote:
  It's a shell script, no need to compile it. Get it here:
  http://cvs.berlios.de/cgi-bin/viewcvs.cgi/*checkout*/jigdo/jigdo/scripts
 /jigdo-lite?rev=HEADcontent-type=text/plain
 
  Can you put multiple (up to 14) --scan switches?
 
  No, but you can make a directory and create 14 symlinks inside it to the
  directories you want scanned.

 Thank you, it works! The files are now in:
http://kambing.vlsm.org/debian-cd/3.1_testing/041207/

 My script is like following:

 
 #!/bin/sh
 for ii in 1 2 3 4 5 6 7 8 9 10 11 12 13 14
 do
   nice -20 jigdo-lite --noask \
--scan /extra/iso/iso/ \
sarge-i386-$ii.jigdo
 done
 exit 0
 


This looks great, but considering that maybe half or more of the updates are 
not really changes but just result from shuffling files from one iso to 
another, is it possible to do something like this:

for ii in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15#yes it is 15 now
do
# loop mount my $iith iso
for jj in 1 ... 15
do
jigdo-lite ...
# scan only the mounted iso
# do not fetch from the internet
done
# unmount the $iith iso
done

# now another loop to update from the net.
...

Any chance?

Regards,
Bob


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite --noask --scan PATH :IT works!

[Richard Atterer]

On Tue, Dec 07, 2004 at 11:28:52PM +1100, Robert Parker wrote:
   jigdo-lite ...
   # scan only the mounted iso
   # do not fetch from the internet

The version of jigdo-lite that only scans files and does not fetch things
from the net is called jigdo-file, i.e. the lower-level tool that
jigdo-lite relies on. ;-)

jigdo-file make-image --image sarge-i386-1.iso \
  --template sarge-i386-1.template /mnt/someiso

Repeat this command for each mounted .iso. After that, start jigdo-lite.

HTH,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite --noask --scan PATH :IT works!

[Robert Parker]

On Wednesday 08 December 2004 00:42, Richard Atterer wrote:

 The version of jigdo-lite that only scans files and does not fetch things
 from the net is called jigdo-file, i.e. the lower-level tool that
 jigdo-lite relies on. ;-)

 jigdo-file make-image --image sarge-i386-1.iso \
   --template sarge-i386-1.template /mnt/someiso

 Repeat this command for each mounted .iso. After that, start jigdo-lite.

That's great. Thanks.

Regards
Bob


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo .template problem

[Steve McIntyre]

On Mon, Oct 18, 2004 at 10:23:51PM +0200, [EMAIL PROTECTED] wrote:
Dnia pon 18. pa?dziernika 2004 16:58, Steve McIntyre napisa?:
 On Mon, Oct 18, 2004 at 11:14:21AM +0200, [EMAIL PROTECTED] wrote:
 Hello
 I'e been trying to download the woody isos through jigdo, bot I keep on
 getting:

 ...

 11:05:30 (42.53 KB/s) - zapisano
 `woody-i386-1_NONUS.template' [29841902/29841902]
 
 /usr/bin/jigdo-file: relocation error: /usr/bin/jigdo-file: symbol
 _ZN9__gnu_cxx18__exchange_and_addEPVii, version GLIBCXX_3.4 not defined in
 file libstdc++.so.6 with link time reference
 Error - template checksum mismatch!
 The .template file does not belong to the .jigdo file - the
 chances are high that the image generation process will break.
 I will abort now. If you know better than me and want this error
 to be ignored, enter the string 42 to proceed.

 Hmmm. The above errors look bad. It looks to me like there's nothing
 necessarily wrong with the templates, but your copy of jigdo-file is
 unhappy. It's looking for versioned symbols from libstdc++.so.6, but
 is not finding them. What system are you running jigdo on, and which
 version of jigdo do you have?

Jigdo version 0.7.0
Mandrake 10.0

Hmmm. Richard? Can you help here?

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
  Armed with Valor: Centurion represents quality of Discipline,
  Honor, Integrity and Loyalty. Now you don't have to be a Caesar to
  concord the digital world while feeling safe and proud.


signature.asc
Description: Digital signature

Re: Jigdo .template problem

[Richard Atterer]

On Thu, Oct 21, 2004 at 04:17:22PM +0100, Steve McIntyre wrote:
  /usr/bin/jigdo-file: relocation error: /usr/bin/jigdo-file: symbol
  _ZN9__gnu_cxx18__exchange_and_addEPVii, version GLIBCXX_3.4 not defined in
  file libstdc++.so.6 with link time reference
[...]
 
 Jigdo version 0.7.0
 Mandrake 10.0
 
 Hmmm. Richard? Can you help here?

Unfortunately not. :-/ This looks like a Mandrake-specific GCC 3.4 messup.

So the only advice I can give to the original poster: Use the jigdo for
Linux which can be downloaded from the jigdo homepage. That binary is
statically linked, so you should not experience problems like the one
above.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo .template problem

[mikiwoz]

Dnia czw 21. padziernika 2004 17:38, Richard Atterer napisa:
 On Thu, Oct 21, 2004 at 04:17:22PM +0100, Steve McIntyre wrote:
   /usr/bin/jigdo-file: relocation error: /usr/bin/jigdo-file: symbol
   _ZN9__gnu_cxx18__exchange_and_addEPVii, version GLIBCXX_3.4 not
defined in file libstdc++.so.6 with link time reference

 [...]

  Jigdo version 0.7.0
  Mandrake 10.0
 
  Hmmm. Richard? Can you help here?

 Unfortunately not. :-/ This looks like a Mandrake-specific GCC 3.4 messup.

 So the only advice I can give to the original poster: Use the jigdo for
 Linux which can be downloaded from the jigdo homepage. That binary is
 statically linked, so you should not experience problems like the one
 above.

 Cheers,

   Richard
A quote from Nirvana Well, whatever, nevermind. 
I've already downloaded the whole debian distro through ftp.  Sorry for it, 
regret soing it, but already done it. :)

Anyway, thanks for your help. If it wasn't the speed of my net connection, I 
would definetely try to download Debian through jigdo.

Cheers, and hope to see you guy on some mailing list when I will be using 
Debian (why does Mandrake try to be smarter then the average user?)

Regards
Mike

Re: Jigdo .template problem

[Steve McIntyre]

On Mon, Oct 18, 2004 at 11:14:21AM +0200, [EMAIL PROTECTED] wrote:
Hello
I'e been trying to download the woody isos through jigdo, bot I keep on 
getting:

...

11:05:30 (42.53 KB/s) - zapisano 
`woody-i386-1_NONUS.template' [29841902/29841902]

/usr/bin/jigdo-file: relocation error: /usr/bin/jigdo-file: symbol 
_ZN9__gnu_cxx18__exchange_and_addEPVii, version GLIBCXX_3.4 not defined in 
file libstdc++.so.6 with link time reference
Error - template checksum mismatch!
The .template file does not belong to the .jigdo file - the
chances are high that the image generation process will break.
I will abort now. If you know better than me and want this error
to be ignored, enter the string 42 to proceed.

Hmmm. The above errors look bad. It looks to me like there's nothing
necessarily wrong with the templates, but your copy of jigdo-file is
unhappy. It's looking for versioned symbols from libstdc++.so.6, but
is not finding them. What system are you running jigdo on, and which
version of jigdo do you have?

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
I've only once written 'SQL is my bitch' in a comment. But that code 
 is in use on a military site... -- Simon Booth


signature.asc
Description: Digital signature

Re: Jigdo .template problem

[mikiwoz]

Dnia pon 18. padziernika 2004 16:58, Steve McIntyre napisa:
 On Mon, Oct 18, 2004 at 11:14:21AM +0200, [EMAIL PROTECTED] wrote:
 Hello
 I'e been trying to download the woody isos through jigdo, bot I keep on
 getting:

 ...

 11:05:30 (42.53 KB/s) - zapisano
 `woody-i386-1_NONUS.template' [29841902/29841902]
 
 /usr/bin/jigdo-file: relocation error: /usr/bin/jigdo-file: symbol
 _ZN9__gnu_cxx18__exchange_and_addEPVii, version GLIBCXX_3.4 not defined in
 file libstdc++.so.6 with link time reference
 Error - template checksum mismatch!
 The .template file does not belong to the .jigdo file - the
 chances are high that the image generation process will break.
 I will abort now. If you know better than me and want this error
 to be ignored, enter the string 42 to proceed.

 Hmmm. The above errors look bad. It looks to me like there's nothing
 necessarily wrong with the templates, but your copy of jigdo-file is
 unhappy. It's looking for versioned symbols from libstdc++.so.6, but
 is not finding them. What system are you running jigdo on, and which
 version of jigdo do you have?

Jigdo version 0.7.0
Mandrake 10.0

Re: jigdo problems (checksum error)

[Steve McIntyre]

On Mon, Oct 11, 2004 at 09:43:27AM +, Russell L. Harris wrote:
Running jigdo-lite/0.7.1 (GNU Wget 1.9.1; linux-gnu), I see many  
(dozens, if not hundreds) instances of checksum errors of the following  
variety:

Error: `/mnt/pool/main/k/krb5/libkrb53_1.3.4-4_i386.deb' does not match  
checksum in template data

I began seeing this error with the .jigdo and .template files I  
downloaded a week ago, and I am seeing the same error on the files I  
downloaded this weekend (the 10th).

Last week, I tried several different CD ISO images from the i386  
archive, but the same problem occurred with all the images I tried to  
download.

There was a bug in the code used to generate the templates. The files
are correct and the templates are wrong. As a temporary workaround,
add --no-check-files to jigdoOpts in your ~/.jigdo-lite.

Manty, can you apply the fix I mailed you before running any more
CD/DVD templates on gluck???

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
This dress doesn't reverse. -- Alden Spiess


signature.asc
Description: Digital signature

Re: jigdo-lite Error: ... does not match checksum in template data

[Achim Lbbert]

Richard Atterer [EMAIL PROTECTED] wrote:
 On Sat, Oct 02, 2004 at 03:12:14PM +0200, Achim Löbbert wrote:
  using yesterday's official sarge templates I am getting tons of errors
  once jigdo-lite starts to write the DVD iso image:
 [...]
  Merging parts from `file:' URIs, if any...
  Found 6720 of the 6720 files required by the template
  Error: 
  `/pub/mirrors/ftp.debian.org/debian/pool/main/a/adduser/adduser_3.59_all.deb' does 
  not match checksum in template data
  Error: `/pub/mirrors/ftp.debian.org/debian/pool/main/a/apt/apt_0.5.27_i386.deb' 
  does not match checksum in template data
 
 As you say later, this error only happens if a file's md5sum changes
 between the initial scan and copying the matching file to the image. :-|

This time I let it run until it wrote 100% of the image. Then it
started to download files in some random, at least non-alphabetic
order. After the first 10 files there is this message:

  FINISHED --16:15:24--
  Downloaded: 4,462,916 bytes in 10 files
  Found 10 of the 5042 files required by the template

which indicates to me that it wants to download 5042 of the 6720 files
required by the template. Of course I stopped it again.

 Apart from undetected read errors on your HD (unlikely...), another 

I compared the files downloaded to a tmp directory with the ones of my
mirror and found them to be identical:

$ cd 
./sarge-i386-1.iso.tmpdir/gluck.debian.org/cdimage/snapshot/dvd/i386/Debian/pool/main
$ find -type f
./g/gxine/gxine_0.3.3-3_i386.deb
./g/gnutls11/gnutls-bin_1.0.16-7_i386.deb
./j/joystick/joystick_20010903-2_i386.deb
./l/lg-issue50/lg-issue50_2-1_all.deb
./s/sc/sc_7.16-2_i386.deb
./libc/libcaca/libcaca-dev_0.9-4_i386.deb
./libh/libhtml-tree-perl/libhtml-tree-perl_3.18-1_all.deb
./libp/libproplist/libproplist0_0.10.1-8_i386.deb
./libt/libtool/libtool-doc_1.5.6-2_all.deb
$ cmp  g/gnutls11/gnutls-bin_1.0.16-7_i386.deb 
\
  /pub/mirrors/ftp.debian.org/debian/pool/main/g/gnutls11/gnutls-bin_1.0.16-7_i386.deb

 reason for this might be a corrupted cache file.

I first tried with a two week old jigdo-file-cache.db in the same
directory. Next time I deleted it prior to running jigdo-lite, both
with the same result.

 You can check whether this is the case by trying
 
   jigdo-file --cache=path/to/cache.db md5sum 
 /pub/mirrors/ftp.debian.org/debian/pool/main/a/adduser/adduser_3.59_all.deb
 
 once with and once without the --cache switch.

$ jigdo-file --cache=jigdo-file-cache.db md5sum 
/pub/mirrors/ftp.debian.org/debian/pool/main/a/adduser/adduser_3.59_all.deb
K3IwZGUJlF7q6sCv7t8PfA  
/pub/mirrors/ftp.debian.org/debian/pool/main/a/adduser/adduser_3.59_all.deb
$ jigdo-file md5sum 
/pub/mirrors/ftp.debian.org/debian/pool/main/a/adduser/adduser_3.59_all.deb
K3IwZGUJlF7q6sCv7t8PfA  
/pub/mirrors/ftp.debian.org/debian/pool/main/a/adduser/adduser_3.59_all.deb

Both checksums look the same.

 This problem with the cache will also appear if something changed the
 content of files in your mirror without also changing the mtime of the
 file. Examples include incorrect mirroring (rsync problems?) or storing the
 mirror on a FAT32 filesystem and enabling its broken CRLF-LF conversion.

The mirror is on a ReiserFS on kernel 2.6.8.1, kept up-to-date by a
modified debmirror. Purging is done by own script with 90 days
latency. The mirror logs show no problems.

  Files in the local mirror [file:/pub/mirrors/ftp.debian.org/debian/] are
  unlikely to have changed between the scan and writing the image. Two
  weeks ago jigdo-lite version 0.7.1-4 did not produce such errors.
 
 And the version that exhibits the problem is also 0.7.1-4?

Yes, I use that version since July 6th.

Achim


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: FIXED: Re: jigdo-lite Error: ... does not match checksum in template data

[Richard Atterer]

On Wed, Oct 06, 2004 at 01:50:32AM +0100, Steve McIntyre wrote:
 I've found the cause of this bug:

Cool, thanks!

 I'm now seeing another minor issue. jigdo-file ls shows
 adduser_3.59_all.deb to have rsyncsum kRjI35yjRzk whereas my own debug
 tools show it to be kRjI35yjRz5 - note the last character is
 changed.

Hmm, I believe your base64 implementation might be buggy. :-/

The base64 checksum is 11 characters long, each character stands for 6
bits. So there are two spare, unused bits (11*6=66). The k stands for 36
(binary 100100), the 5 for 57 (binary 111001).

Base64 is big-endian per byte. In other words, 3*8 input bits in
big-endian order are subdivided into 4*6 output bits, like this:

07 06 05 04 03 02 01 00|15 14 13 12 11 10 09 08|23 22 21 20 19 18 17 16
07 06 05 04 03 02|01 00 15 14 13 12|11 10 09 08 23 22|21 20 19 18 17 16
 jigdo-file: 1  0  0  1  0  0
JTE: 1  1  1  0  0  1

In this case, input bits 16..23 are non-existent (because the input is not
a multiple of 3 bytes). The unused bits 22 and 23 are set to 0 by
jigdo-file. For jigdo-file, bits 8..11 are set to 1001, which matches the
last hex character of the hex checksum 9118c8df9ca34739.

I just compared our two implementations of base64. Did you use my code as a
reference for your work or is there only one way to implement it??  Anyway,
the algorithm is almost exactly the same except for some tiny differences. 
Applying the patch below (untested, sorry) might make a difference.

IIRC, when I implemented my base64, I checked whether the above 
interpretation of the RFC was correct by encoding binary data with some 
mail client.

 Richard, PLEASE for future versions of jigdo drop this silly bastardised
 base64 encoding that you're using. It makes life _very_ difficult when
 debugging things if the checksum output format is not simple. Base 16
 good, base 64 bad.

Um, I'll think about it, but I don't see a problem here. IMHO the only
thing that's more difficult is the implementation of the binary-baseX
conversion, so if /that/ is correct, everything should be fine... :-/

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯

--- jte.c.orig  2004-10-06 11:34:55.0 +0200
+++ jte.c   2004-10-06 11:43:58.0 +0200
@@ -765,15 +765,15 @@
 value = (value  8) | buf[i];
 bits += 2;
 p += sprintf(p, %c, b64_enc[(value  bits)  63U]);
-if (bits = 8) {
+if (bits = 6) {
 bits -= 6;
 p += sprintf(p, %c, b64_enc[(value  bits)  63U]);
 }
 }
 if (bits  0)
 {
-value = 8 - bits;
-p += sprintf(p, %c, b64_enc[(value  bits)  63U]);
+value = 6 - bits;
+p += sprintf(p, %c, b64_enc[value  63U]);
 }
 return output_buffer;
 }


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: FIXED: Re: jigdo-lite Error: ... does not match checksum in template data

[Steve McIntyre]

On Wed, Oct 06, 2004 at 11:49:45AM +0200, Richard Atterer wrote:
On Wed, Oct 06, 2004 at 01:50:32AM +0100, Steve McIntyre wrote:

 I'm now seeing another minor issue. jigdo-file ls shows
 adduser_3.59_all.deb to have rsyncsum kRjI35yjRzk whereas my own debug
 tools show it to be kRjI35yjRz5 - note the last character is
 changed.

Hmm, I believe your base64 implementation might be buggy. :-/

...

I just compared our two implementations of base64. Did you use my code as a
reference for your work or is there only one way to implement it??  Anyway,
the algorithm is almost exactly the same except for some tiny differences. 
Applying the patch below (untested, sorry) might make a difference.

Correct. Your patch fixes it. A base64 algorithm is always going to be
rather similar... :-)

 Richard, PLEASE for future versions of jigdo drop this silly bastardised
 base64 encoding that you're using. It makes life _very_ difficult when
 debugging things if the checksum output format is not simple. Base 16
 good, base 64 bad.

Um, I'll think about it, but I don't see a problem here. IMHO the only
thing that's more difficult is the implementation of the binary-baseX
conversion, so if /that/ is correct, everything should be fine... :-/

The problem is that the rest of the world outputs large binary data
into text format by using hexadecimal; it's easy to understand and
much easier to output and (more importantly) parse in a variety of
languages. The only reason I can see in favour of the base64 encoding
is a reduction in output size, and I'm not convinced that the small
gains are worth the hassle. Please consider moving to the more
standard format... :-P

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
Is there anybody out there?


signature.asc
Description: Digital signature

Re: jigdo-lite Error: ... does not match checksum in template data

[vex127]

Downloaders, as a short-term workaround add
--no-check-files to jigdoOpts in your
~/.jigdo-lite.

Okay did that and still get errors... fixed next week?

__
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

FIXED: Re: jigdo-lite Error: ... does not match checksum in template data

[Steve McIntyre]

On Sat, Oct 02, 2004 at 06:52:40PM +0200, Richard Atterer wrote:
OK, I know now what causes the problem. I haven't found the error in JTE 
(or jigdo-file?) yet.

Downloaders, as a short-term workaround add --no-check-files to jigdoOpts 
in your ~/.jigdo-lite.

As of version 1.12, JTE not only outputs md5sums for each file, but also an
Rsync64Sum for the first 1k of each file. I asked Steve to implement this
- in the future, it'll allow the jigdo GUI to abort downloads early if the
checksum of the first 1k doesn't match during the download.

JTE 1.11 outputs a zero blockLen field to the .template, which switches
off comparison of Rsync64Sums by jigdo-file make-image (used by
jigdo-lite).

JTE 1.12 sets blockLen to 1024, but outputs a wrong Rsync64Sum. With our 
example file, adduser_3.59_all.deb:

$ jigdo-file mt -fi adduser_3.59_all.deb -j grr.jigdo -t grr.template 
adduser_3.59_all.deb
Match of `adduser_3.59_all.deb' at offset 0   
Finished - image size is 93882 bytes.
$ jigdo-file ls -t grr.template
need-file  093882 K3IwZGUJlF7q6sCv7t8PfA kRjI35yjRzk
image-info 93882  K3IwZGUJlF7q6sCv7t8PfA 1024
$ jigdo-file ls -t grr.template --hex
need-file  093882 2b7230646509945eeaeac0afeedf0f7c 
9118c8df9ca34739
image-info 93882  2b7230646509945eeaeac0afeedf0f7c 1024

This means that the Rsync64Sum expected by jigdo-file is kRjI35yjRzk, or 
9118c8df9ca34739 in hexadecimal. For the algorithm, this means that 
lo=0xdfc81891, hi=0x3947a39c.

I've found the cause of this bug:

==
--- jte.c~  2004-09-05 17:35:53.0 +0100
+++ jte.c   2004-10-06 01:07:48.0 +0100
@@ -944,6 +944,7 @@
FILE   *infile = NULL;
int use = 0;
 unsigned long long  rsync64_sum = 0;
+int first_block = 1;
 
 memset(buf, 0, sizeof(buf));
 
@@ -964,8 +965,6 @@
 
 while (remain  0)
 {
-int first_block = 1;
-
 use = remain;
 if (remain  sizeof(buf))
 use = sizeof(buf);
==

fixes it. I was resetting first_block for every block :-(

Manty, can you please apply this to your copy of JTE before the next
DVD build please?

I'm now seeing another minor issue. jigdo-file ls shows
adduser_3.59_all.deb to have rsyncsum kRjI35yjRzk whereas my own debug
tools show it to be kRjI35yjRz5 - note the last character is
changed. The algorithm I'm using for printing the base64-encoded
numbers is the same as for the md5sum results, so I'm bothered by this
being wrong. I also can't see any mistake in the code.

Richard, PLEASE for future versions of jigdo drop this silly
bastardised base64 encoding that you're using. It makes life _very_
difficult when debugging things if the checksum output format is not
simple. Base 16 good, base 64 bad.

/rant.

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
Support the Campaign for Audiovisual Free Expression: http://www.eff.org/cafe/


signature.asc
Description: Digital signature

Re: jigdo-lite Error: ... does not match checksum in template data

[Mattias Wadenstein]

On Sun, 3 Oct 2004, Richard Atterer wrote:
My goodness - Manty or Mattias, can you please disable access to the broken
DVD jigdo files, or at least put a big README into the directory which
explains that they're broken?
Ok, I've put in a temporary README on cdimage.d.o. I'll look at disabling 
access later on.

/Mattias Wadenstein
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite Error: ... does not match checksum in template data

[Richard Atterer]

My goodness - Manty or Mattias, can you please disable access to the broken 
DVD jigdo files, or at least put a big README into the directory which 
explains that they're broken?

I'm getting quite a few private mails from people complaining about
jigdo-lite failure... :-/

Thanks,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite Error: ... does not match checksum in template data

[Richard Atterer]

On Sat, Oct 02, 2004 at 03:12:14PM +0200, Achim Löbbert wrote:
 using yesterday's official sarge templates I am getting tons of errors
 once jigdo-lite starts to write the DVD iso image:
[...]
 Merging parts from `file:' URIs, if any...
 Found 6720 of the 6720 files required by the template
 Error: `/pub/mirrors/ftp.debian.org/debian/pool/main/a/adduser/adduser_3.59_all.deb' 
 does not match checksum in template data
 Error: `/pub/mirrors/ftp.debian.org/debian/pool/main/a/apt/apt_0.5.27_i386.deb' does 
 not match checksum in template data

As you say later, this error only happens if a file's md5sum changes
between the initial scan and copying the matching file to the image. :-|

Apart from undetected read errors on your HD (unlikely...), another 
reason for this might be a corrupted cache file. You can check whether this 
is the case by trying

  jigdo-file --cache=path/to/cache.db md5sum 
/pub/mirrors/ftp.debian.org/debian/pool/main/a/adduser/adduser_3.59_all.deb

once with and once without the --cache switch.

This problem with the cache will also appear if something changed the
content of files in your mirror without also changing the mtime of the
file. Examples include incorrect mirroring (rsync problems?) or storing the
mirror on a FAT32 filesystem and enabling its broken CRLF-LF conversion.

 Files in the local mirror [file:/pub/mirrors/ftp.debian.org/debian/] are
 unlikely to have changed between the scan and writing the image. Two
 weeks ago jigdo-lite version 0.7.1-4 did not produce such errors.

And the version that exhibits the problem is also 0.7.1-4?

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite Error: ... does not match checksum in template data

[Markus Hirschmann]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi Achim
Achim Löbbert wrote:
| using yesterday's official sarge templates I am getting tons of errors
| once jigdo-lite starts to write the DVD iso image:
Same Problem here.
I think there's a problem in the jidgo-templates, last weeks templates
worked perfectly.
Greetings
Markus
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFBXsEw7DLmG4oCQnQRAgMYAJ9ibnNH8iNwY3V1S/x3+nMk1sybFACcCeXX
rW2f2s/X6CIvz11Krgqb6vA=
=qiDf
-END PGP SIGNATURE-

Re: jigdo-lite Error: ... does not match checksum in template data

[Richard Atterer]

OK, I know now what causes the problem. I haven't found the error in JTE 
(or jigdo-file?) yet.

Downloaders, as a short-term workaround add --no-check-files to jigdoOpts 
in your ~/.jigdo-lite.

As of version 1.12, JTE not only outputs md5sums for each file, but also an
Rsync64Sum for the first 1k of each file. I asked Steve to implement this
- in the future, it'll allow the jigdo GUI to abort downloads early if the
checksum of the first 1k doesn't match during the download.

JTE 1.11 outputs a zero blockLen field to the .template, which switches
off comparison of Rsync64Sums by jigdo-file make-image (used by
jigdo-lite).

JTE 1.12 sets blockLen to 1024, but outputs a wrong Rsync64Sum. With our 
example file, adduser_3.59_all.deb:

$ jigdo-file mt -fi adduser_3.59_all.deb -j grr.jigdo -t grr.template 
adduser_3.59_all.deb
Match of `adduser_3.59_all.deb' at offset 0   
Finished - image size is 93882 bytes.
$ jigdo-file ls -t grr.template
need-file  093882 K3IwZGUJlF7q6sCv7t8PfA kRjI35yjRzk
image-info 93882  K3IwZGUJlF7q6sCv7t8PfA 1024
$ jigdo-file ls -t grr.template --hex
need-file  093882 2b7230646509945eeaeac0afeedf0f7c 9118c8df9ca34739
image-info 93882  2b7230646509945eeaeac0afeedf0f7c 1024

This means that the Rsync64Sum expected by jigdo-file is kRjI35yjRzk, or 
9118c8df9ca34739 in hexadecimal. For the algorithm, this means that 
lo=0xdfc81891, hi=0x3947a39c.

The JTE-generated .template contains something else:

$ jigdo-file ls -t sarge-i386-1.template|grep 52803584
need-file   5280358493882 K3IwZGUJlF7q6sCv7t8PfA gIOeQJMjgK4
$ jigdo-file ls -t sarge-i386-1.template --hex|grep 52803584
need-file   5280358493882 2b7230646509945eeaeac0afeedf0f7c 80839e40932380ae

So for JTE, lo=0x409e8380 and hi=0xae802393 - the bytes are not just
reordered like I initially suspected, but completely different. %-| Having
stared at JTE's implementation of the algorithm as well as jigdo-file's (in
src/rsyncsum.cc:73), I cannot figure out any differences. Grr, I'll have a
closer look tomorrow - unless Steve beats me in finding the bug. ;-)

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo-lite Error: ... does not match checksum in template data

[Steve McIntyre]

On Sat, Oct 02, 2004 at 06:52:40PM +0200, Richard Atterer wrote:

$ jigdo-file mt -fi adduser_3.59_all.deb -j grr.jigdo -t grr.template 
adduser_3.59_all.deb
Match of `adduser_3.59_all.deb' at offset 0   
Finished - image size is 93882 bytes.
$ jigdo-file ls -t grr.template
need-file  093882 K3IwZGUJlF7q6sCv7t8PfA kRjI35yjRzk
image-info 93882  K3IwZGUJlF7q6sCv7t8PfA 1024
$ jigdo-file ls -t grr.template --hex
need-file  093882 2b7230646509945eeaeac0afeedf0f7c 
9118c8df9ca34739
image-info 93882  2b7230646509945eeaeac0afeedf0f7c 1024

This means that the Rsync64Sum expected by jigdo-file is kRjI35yjRzk, or 
9118c8df9ca34739 in hexadecimal. For the algorithm, this means that 
lo=0xdfc81891, hi=0x3947a39c.

The JTE-generated .template contains something else:

$ jigdo-file ls -t sarge-i386-1.template|grep 52803584
need-file   5280358493882 K3IwZGUJlF7q6sCv7t8PfA gIOeQJMjgK4
$ jigdo-file ls -t sarge-i386-1.template --hex|grep 52803584
need-file   5280358493882 2b7230646509945eeaeac0afeedf0f7c 
80839e40932380ae

So for JTE, lo=0x409e8380 and hi=0xae802393 - the bytes are not just
reordered like I initially suspected, but completely different. %-| Having
stared at JTE's implementation of the algorithm as well as jigdo-file's (in
src/rsyncsum.cc:73), I cannot figure out any differences. Grr, I'll have a
closer look tomorrow - unless Steve beats me in finding the bug. ;-)

I can't promise anything at the moment; I'm stuck away from home with
a family emergency right now. I tested this, so I'm surprised it
doesn't work... :-(

I'll see if I can take a look.

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
We don't need no education.
We don't need no thought control.


signature.asc
Description: Digital signature

Re: jigdo problem .bug?

[ÌÜêçò]

Hmm you didnt  help a lot..
Yes before i sent you this mail i sure checked all things you say(entered
the
failed URL in to the browser,checked proxy settings), anyway the ftp was
working ,had only problem with http(having the message i copied below)

and the solution was to downgrade jigdo-file from testing distribution to
stable()From version(0.7.1-5 to 0.6.5-2). Now works fine!

anyway thanks

On Fri, 24 Sep 2004, Richard Atterer wrote:

 On Fri, Sep 24, 2004 at 04:43:57PM +0300, Euthimis Drimwnas wrote:
  im trying to download the dvd images just as you say in the instructions,
  but when some files remain(150-160 i think) jigdo tries to download the
  remaining from http://gluck.debian and then wget says :
 
  HTTP request sent, awaiting response...
  Read error (Connection reset by peer) in headers.
  Retrying.
 
  i noticed that it tries to download some packages (ex webmin- version 1.50
  but  1.50 exists only in gluck, the others have 1.60) but from http
  location is not working gets the previous error.

 How do you know that webmin 1.50 only exists on gluck? At a guess you
 entered the failed wget URL into your browser to check? In that case it is
 very likely that your proxy settings for jigdo-lite are not correct - see
 http://www.debian.org/CD/jigdo-cd/#faq on how to set them.

 Cheers,

   Richard

 --
   __   _
   |_) /|  Richard Atterer |  GnuPG key:
   | \/¯|  http://atterer.net  |  0x888354F7
   ¯ '` ¯

Re: jigdo problem .bug?

[Richard Atterer]

On Fri, Sep 24, 2004 at 04:43:57PM +0300, Euthimis Drimwnas wrote:
 im trying to download the dvd images just as you say in the instructions,
 but when some files remain(150-160 i think) jigdo tries to download the
 remaining from http://gluck.debian and then wget says :
 
 HTTP request sent, awaiting response...
 Read error (Connection reset by peer) in headers.
 Retrying.
 
 i noticed that it tries to download some packages (ex webmin- version 1.50
 but  1.50 exists only in gluck, the others have 1.60) but from http
 location is not working gets the previous error.

How do you know that webmin 1.50 only exists on gluck? At a guess you 
entered the failed wget URL into your browser to check? In that case it is 
very likely that your proxy settings for jigdo-lite are not correct - see 
http://www.debian.org/CD/jigdo-cd/#faq on how to set them.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo says Aaargh - 1 files could not be download, even when the file has been saved

[José Ramón Rodrigo Roldán]

 --- Richard Atterer [EMAIL PROTECTED]
wrote: 
 Hi Jose,

Hi, Richard.
 
 this appears to be a problem at your end, most
 likely with your HTTP proxy. 
 Please enter this URL into your browser and check
 the error message that 
 you get instead of the file:
 
  ool/main/g/gnome-sudo/gnome-sudo_0.3-1_i386.deb'
  Connecting to 192.168.113.1:8080... connected.
  Proxy request sent, awaiting response... 200 OK
  Length: unspecified [text/html]
   ^^^
 Here, jigdo tries to get a Debian package, but the
 server sends a response
 which is HTML and only 504 bytes short:

 The same thing happens here, with the Debian backup
 server. I've verified
 this also holds the correct data. I suspect your
 HTTP proxy performs some
 filtering... is sudo a dirty word in Spanish? :-)
 
No, it isn't. And I can assure you that I have a big
bad vocabulary set... But it was as you said: i tried
another net connection without proxies and it worked.
Well, actually I downloaded first the file, and then
rescaned the isos, as Gordon adviced me before.

I don't known why among almost five thounsand files
this lonely file didn't pass the proxy. Maybe Sudo
is equal to Sado. ¡¡Then I have to load that package
in the installation!!

Thank you very much.



__
Renovamos el Correo Yahoo!: ¡100 MB GRATIS!
Nuevos servicios, más seguridad
http://correo.yahoo.es


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: Jigdo says Aaargh - 1 files could not be download, even when the file has been saved

[Richard Atterer]

Hi Jose,

this appears to be a problem at your end, most likely with your HTTP proxy. 
Please enter this URL into your browser and check the error message that 
you get instead of the file:

http://us.cdimage.debian.org/jigdo-area/3.0_r2/snapshot/pool/main/g/gnome-sudo/gnome-sudo_0.3-1_i386.deb

On Fri, Aug 27, 2004 at 02:15:46AM +0200, Josÿe9 
Ramÿf3n Rodrigo wrote:
 --02:04:26-- 
 http://www.furryterror.org/debian/pool/main/g/gnome-sudo/gnome-sud
 o_0.3-1_i386.deb
=
 `debian-30r2-i386-binary-1.iso.tmpdir/www.furryterror.org/debian/p
 ool/main/g/gnome-sudo/gnome-sudo_0.3-1_i386.deb'
 Connecting to 192.168.113.1:8080... connected.
 Proxy request sent, awaiting response... 200 OK
 Length: unspecified [text/html]
  ^^^
Here, jigdo tries to get a Debian package, but the server sends a response
which is HTML and only 504 bytes short:

 [ = ] 504   
--.--K/s

When I try this from here (with the same server), I get:
  Length: 12,696 [application/x-debian-package]
and the correct data.

 http://us.cdimage.debian.org/jigdo-area/3.0_r2/snapshot/pool/main/
 g/gnome-sudo/gnome-sudo_0.3-1_i386.deb
=
 `debian-30r2-i386-binary-1.iso.tmpdir/us.cdimage.debian.org/jigdo-
 area/3.0_r2/snapshot/pool/main/g/gnome-sudo/gnome-sudo_0.3-1_i386.deb'
 Connecting to 192.168.113.1:8080... connected.
 Proxy request sent, awaiting response... 200 OK
 Length: unspecified [text/html]
 
 [ = ] 526   

The same thing happens here, with the Debian backup server. I've verified
this also holds the correct data. I suspect your HTTP proxy performs some
filtering... is sudo a dirty word in Spanish? :-)

 I gave to jidgo several URLs, with the same result. It's strange, because
 jidgo says that the file has been saved and a couple lines later affirms
 that it has not been found.

jigdo calculates the checksum of the downloaded data and compares it to a 
checksum supplied in the .template file. The data is only accepted if the 
checksums match, which was not the case above.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo and debian-cd

[Lucio Crusca]

Richard Atterer wrote:
 jigdo-mirror is intended for this job.

Thanks Richard, but there is something I didn't quite catch about 
jigdo-mirror:

Does it check for already downloaded debs in a particular directory? Or does 
it mount -o loop any ISO image it finds? Or does it fetch the full ISO images 
everytime? Does it create new ISO images just the same way as jigdo-lite 
does?

Lucio.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo and debian-cd

[Richard Atterer]

On Fri, Jun 18, 2004 at 10:42:21AM +0200, Lucio Crusca wrote:
 Does it check for already downloaded debs in a particular directory? Or
 does it mount -o loop any ISO image it finds?

Neither one - it assumes you either have a full Debian mirror on your hard
disc or in your LAN.

 Or does it fetch the full ISO images everytime?

Yes; it assumes that the individual packages can be retrieved very fast.

 Does it create new ISO images just the same way as jigdo-lite does?

Apart from the above points, yes.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo and debian-cd

[Richard Atterer]

On Thu, Jun 17, 2004 at 03:16:17PM +0200, Lucio Crusca wrote:
 1 - What should I use, debian-cd or jigdo? 

jigdo. :)

 1.1 - Just in case the answer is jigdo, is there a way to make the process 
 of updating images less interactive (not interactive at all) so that 
 jigdo-lite can be put in a shell script?
 1.1.1 - Does this shell script already exist somewhere?

jigdo-mirror is intended for this job. Alternatively, someone posted a 
patch for jigdo-lite just the other day:

https://lists.berlios.de/pipermail/jigdo-user/2004-June/000431.html

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo and unofficial Debian images

[Bob Proulx]

Richard Atterer wrote:
 [EMAIL PROTECTED] wrote:
  The most common problem with jigdo I've found is that what is specified
  in the .jigdo file to be downloaded to complete the image is not an
  accurate reflection of what actually exists on the corresponding mirror. 
 
 This is a general problem with Debian mirrors: They are a moving target - 
 while CD images are only created weekly, the mirror contents change every 
 day.

Would it be possible (or even reasonable given load usage constraints)
to base the unstable jigdo images off of the snapshot.debian.net
archive?  That provides an unmoving snapshot of sid.  I would hate to
overload their archive, however.  It is a very valuable resource and I
use it often but I think it is a single distribution point.  Perhaps
as a fallback only?

Just a passing thought...

Bob


pgpXMBHMFTOmi.pgp
Description: PGP signature

Re: jigdo and unofficial Debian images

[Richard Atterer]

On Sat, Jun 12, 2004 at 01:08:13PM -0600, Bob Proulx wrote:
 Would it be possible (or even reasonable given load usage constraints) to
 base the unstable jigdo images off of the snapshot.debian.net archive? 
 That provides an unmoving snapshot of sid.  I would hate to overload
 their archive, however.  It is a very valuable resource and I use it
 often but I think it is a single distribution point.  Perhaps as a
 fallback only?

Yes, I think using it as a _secondary_ fallback is a good idea. Maybe the
fsn.hu images will soon switch to using it this way.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo vs binary diff

[George Danchev]

On Thursday 10 June 2004 01:35, Richard Atterer wrote:
 On Wed, Jun 09, 2004 at 08:45:05PM +0300, George Danchev wrote:
  what advantages I'll get when using jigdo to update my old image instead
  of patching the old iso with rdiff.

 Well, Debian doesn't offer CD image upgrades in rdiff/xdelta format.
 Generally, IMHO jigdo has the following advantages over binary diffs:

 - jigdo allows you to update to your image from /any/ previous version of
   the same image. That's most useful with weekly snapshots - imagine having
   to download and apply 6 patches because you last fetched a weekly
   snapshot 6 weeks ago.

 - you can download all the changed/new packages from any Debian mirror, not
   just designated CD patch mirrors. With patches, IMHO many regular
   Debian mirrors would not be too excited about mirroring large patch
   files, so the few mirrors who offered them would be fairly busy.

 - related to the point above: Debian does not need to duplicate the data
   for the changed/new packages on the servers in a binary diff, which saves
   space. (jigdo can download these packages from the normal archive of
   packages on a Debian server.)

 - jigdo allows completely different kinds of updates, e.g. you can update
   from a set of CD images to one DVD image, without downloading all the
   package data again.

 - jigdo updates still work fine in case your old CD has read errors.

Thanks for pointing these out. Your explanation is more that sufficient. I 
would add one more advantage of jigdo over rdiff:
- The new image might be piped instead of generated and stored on the server 
from debian-cd (or whatever) to jigdo-file to create the .jigdo/.template 
files. With rdiff one need to have the old and new image files to create the 
diff.

I think this comparing information is very important to clear the things up 
for the users and must be added to the jigdo's site and docs.

-- 
pub 4096R/0E4BD0AB  2003-03-18  keyserver.bu.edu ; pgp.mit.edu
fingerprint 1AE7 7C66 0A26 5BFF DF22 5D55 1C57 0C89 0E4B D0AB 


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo CD directory contains DVD images

[Santiago Garcia Mantinan]

 it seems a mistake was made on the primary mirror for CD jigdo files - the 

Yes, my fault, should be solved now!

Regards...
-- 
Manty/BestiaTester - http://manty.net


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Re: jigdo DVD support under Windows

[Richard Atterer]

On Sun, Feb 29, 2004 at 04:21:59PM -0800, Gordon Huff wrote:
 Let me take the liberty of elaborating on this  ...the big file problem 
 occurs a coupla places in Windows 
 
 For every M$ file system except NTFS, the size of the file [...]

Right - jigdo DVD support on Windows would be NTFS-only.

 Much software uses 32-bit integers  unsigned with a 4 GB limit [...]

Internally, jigdo uses 64-bit integers for all size-related variables, so 
this is not a problem for it.

 Is there anybody out there willing to maintain the Windows port of
 jigdo? It should mostly compile with the latest mingw toolchain, and
 that might also fix the above problem...
 
 Last time I looked  last week ... I did not see where mingw has fixed
 this ...

When I looked into this (long ago, 1 year), I think there *was* big file
support in mingw (at least there were special 64-bit versions of some
system calls), but libstdc++ did not have 64-bit support.

As I found out a while ago, libstdc++'s big file support is also broken in
all GCC 3.x releases so far, I don't know if this applies to the mingw port 
as well. Bug 22 mentions that GCC 3.4 will fix the problem.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]