Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On Thu, Apr 24, 2014 at 4:04 PM, Andreas Tille wrote:
Any idea how we could make uscan capable to detect and download the latest
source anyway?
Use a match based on the zip download and downloadurlmangle to
transform that to a src tarball.
a href=/download/mkgmap-r3226.zip data-bind=attr: {href:
current().full_name}
--
bye,
pabs
http://wiki.debian.org/PaulWise
--
To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive:
https://lists.debian.org/CAKTje6FtVDHCgYCSSt1T7OWdUh=7zsdqabh70owexy_jyh0...@mail.gmail.com
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On 24/04/14 09:04, Andreas Tille wrote: Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. Using something like this would seem to work, added to the opts downloadurlmangle=s/\.zip/-src.tar.gz/ Regards, -- Daniel Lintott GPG Key: 4096R/5D73EC6E signature.asc Description: OpenPGP digital signature
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On 24/04/14 09:44, Daniel Lintott wrote: On 24/04/14 09:04, Andreas Tille wrote: Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. Using something like this would seem to work, added to the opts downloadurlmangle=s/\.zip/-src.tar.gz/ Regards, Having just tried this again.. it only half works... Output from uscan --verbose -- Scanning for watchfiles in . -- Found watchfile in ./debian -- In debian/watch, processing watchfile line: opts=dversionmangle=s/0.0.0.svn//,downloadurlmangle=s/\.zip/-src.tar.gz/ http://www.mkgmap.org.uk/download/ mkgmap-r([\d]+)\.zip -- Found the following matching hrefs: /download/mkgmap-r3226.zip (3226) Newest version on remote site is 3226, local version is 0.0.0+svn2981 (mangled local version number 2981) = Newer version available from http://www.mkgmap.org.uk/download/mkgmap-r3226-src.tar.gz -- Downloading updated package mkgmap-r3226.zip -- Scan finished So it seems as if uscan is ignoring the downloadurlmangle when it comes to actually downloading the file! Regards, -- Daniel Lintott GPG Key: 4096R/5D73EC6E signature.asc Description: OpenPGP digital signature
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On 24/04/14 10:32, Daniel Lintott wrote: On 24/04/14 09:44, Daniel Lintott wrote: On 24/04/14 09:04, Andreas Tille wrote: Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. Using something like this would seem to work, added to the opts downloadurlmangle=s/\.zip/-src.tar.gz/ Regards, Having just tried this again.. it only half works... Output from uscan --verbose -- Scanning for watchfiles in . -- Found watchfile in ./debian -- In debian/watch, processing watchfile line: opts=dversionmangle=s/0.0.0.svn//,downloadurlmangle=s/\.zip/-src.tar.gz/ http://www.mkgmap.org.uk/download/ mkgmap-r([\d]+)\.zip -- Found the following matching hrefs: /download/mkgmap-r3226.zip (3226) Newest version on remote site is 3226, local version is 0.0.0+svn2981 (mangled local version number 2981) = Newer version available from http://www.mkgmap.org.uk/download/mkgmap-r3226-src.tar.gz -- Downloading updated package mkgmap-r3226.zip -- Scan finished So it seems as if uscan is ignoring the downloadurlmangle when it comes to actually downloading the file! Regards, After digging into even more... hate to leave a problem unsolved... I think I cracked it! In addition to the the downloadurlmangle, a filenamemangle is needed to save the downloaded file with the correct name. filenamemangle=s/\/download\/mkgmap-r([\d]+)\.zip/mkgmap-r$1-src.tar.gz/ Which removes the download directory and names the file correctly as a tar.gz Hope that helps... Regards, -- Daniel Lintott GPG Key: 4096R/5D73EC6E signature.asc Description: OpenPGP digital signature
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
Hi Daniel, thanks for your continuous trials to help me. :-) On Thu, Apr 24, 2014 at 11:22:22AM +0100, Daniel Lintott wrote: On 24/04/14 10:32, Daniel Lintott wrote: On 24/04/14 09:44, Daniel Lintott wrote: On 24/04/14 09:04, Andreas Tille wrote: Hi, if you look at the HTML code of http://www.mkgmap.org.uk/download/ you see strings like: \mkgmap-r3226-src.tar.gz\: 1555754, \mkgmap-r3226-src.zip\: 2194070, and thus uscan stopped working. I guess it will be hard to convince upstream that their new shiny page is a bad idea for Debian and that they change back. Any idea how we could make uscan capable to detect and download the latest source anyway? Kind regards Andreas. Using something like this would seem to work, added to the opts downloadurlmangle=s/\.zip/-src.tar.gz/ Regards, Having just tried this again.. it only half works... Output from uscan --verbose -- Scanning for watchfiles in . -- Found watchfile in ./debian -- In debian/watch, processing watchfile line: opts=dversionmangle=s/0.0.0.svn//,downloadurlmangle=s/\.zip/-src.tar.gz/ http://www.mkgmap.org.uk/download/ mkgmap-r([\d]+)\.zip -- Found the following matching hrefs: /download/mkgmap-r3226.zip (3226) Newest version on remote site is 3226, local version is 0.0.0+svn2981 (mangled local version number 2981) = Newer version available from http://www.mkgmap.org.uk/download/mkgmap-r3226-src.tar.gz -- Downloading updated package mkgmap-r3226.zip -- Scan finished So it seems as if uscan is ignoring the downloadurlmangle when it comes to actually downloading the file! Regards, After digging into even more... hate to leave a problem unsolved... I think I cracked it! In addition to the the downloadurlmangle, a filenamemangle is needed to save the downloaded file with the correct name. filenamemangle=s/\/download\/mkgmap-r([\d]+)\.zip/mkgmap-r$1-src.tar.gz/ Which removes the download directory and names the file correctly as a tar.gz Hope that helps... Not really since I can not confirm that I get anything else but: no matching hrefs for watch line I guess we somehow need to go with Paul's hint. All your hints are based on changing the regexp for the source download by droping the '-src' part from the file name. Since we really want to fetch the source and not simply the binary jar this is not helpful. However, I somehow need to make some sense out of Paul's hint which I did not fully understand admittedly. Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140424113214.gi11...@an3as.eu
Re: watch file help needed: Download page of mkgmap changed and is now hiding download files behind JavaScript code
On 24/04/14 12:32, Andreas Tille wrote: Hi Daniel, thanks for your continuous trials to help me. :-) No problem! I enjoy a good bit of problem solving! On Thu, Apr 24, 2014 at 11:22:22AM +0100, Daniel Lintott wrote: Hope that helps... Not really since I can not confirm that I get anything else but: no matching hrefs for watch line I guess we somehow need to go with Paul's hint. All your hints are based on changing the regexp for the source download by droping the '-src' part from the file name. Since we really want to fetch the source and not simply the binary jar this is not helpful. However, I somehow need to make some sense out of Paul's hint which I did not fully understand admittedly. What Paul was hinting at was using using the downloadurlmangle as per my first email. What happens here is we get the href for the binary download and then mangle it to get the source tarball, by replacing .zip with -src.tar.gz s/\.zip/-src.tar.gz/ Next we have to make uscan save the file it downloads with the correct filename... filenamemangle. filenamemangle=s/\/download\/mkgmap-r([\d]+)\.zip/mkgmap-r$1-src.tar.gz/ Here we replace /download/mkgmap-r.zip with mkgmap-r-src.tar.gz to match the name of the that is actually downloaded. I have also attached a copy of the watch file I used to test the mangles. Regards, -- Daniel Lintott GPG Key: 4096R/5D73EC6E version=3 opts=dversionmangle=s/0.0.0.svn//,downloadurlmangle=s/\.zip/-src.tar.gz/,filenamemangle=s/\/download\/mkgmap-r([\d]+)\.zip/mkgmap-r$1-src.tar.gz/ \ http://www.mkgmap.org.uk/download/mkgmap-r([\d]+).zip signature.asc Description: OpenPGP digital signature
Re: Watch file help needed
Andreas Tille andr...@an3as.eu writes: but this does not work as expected. Any hint? 1. you have an extra . in your watchfile. The line should be http://blog.velocarte66.fr/?q=de/node/170 /sites/blog.velocarte66.fr/files//filepicker/1/rel2gpx_([v.\d]+)\.tgz instead of http://blog.velocarte66.fr/?q=de/node/170 /sites/blog.velocarte66.fr/files//filepicker/1/rel2gpx_([v.\d]+)\..tgz 2. The download links from that website are broken (they return a 404 not found error) -- Marius Gavrilescu pgp_HU4boW93V.pgp Description: PGP signature
Re: Watch file help needed
Hi Andreas, At the end of your line, you have this \..tgz, but it's \.tgz And the link http://blog.velocarte66.fr/sites/blog.velocarte66.fr/files//filepicker/1/rel2gpx_v027.tgz on the page is not found. Best regards. -- Cordialement, Blanc Pierre -- To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caogtle+5evfz2q+s5+1dg247dqwwjw252gqox10e9oh5atd...@mail.gmail.com
Re: Watch file help needed
On Sun, Oct 20, 2013 at 07:27:40PM +0200, Andreas Tille wrote: Hi, rel2gpx was moved to a new location http://blog.velocarte66.fr/?q=de/node/170 I tried to adapt the watch file to http://anonscm.debian.org/gitweb/?p=pkg-grass/rel2gpx.git;a=commitdiff;h=HEAD but this does not work as expected. Any hint? Yes, I suggest to use this : | version=3 | opts=uversionmangle=s/^(0)(\d\d)$/$1.$2/ \ | http://blog.velocarte66.fr/?q=de/node/170 \ | (?:|.*/)rel2gpx(?:[_\-]v?|)(\d[^\s/]*)\.(?:tar\.xz|txz|tar\.bz2|tbz2|tar\.gz|tgz) Regards, Bart Martens -- To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131020180844.ga19...@master.debian.org
Re: Watch file help needed
On Wed, Feb 13, 2013 at 09:58:28PM +0600, Andrey Rahmatullin wrote: For the reference: opts=uversionmangle=s/_/./ \ ftp://ftp.ebi.edu.au/pub/software/MEME/index.html .*/meme_([\d\._]+)\.tar\.gz -- Downloading updated package meme_4.9.0_4.tar.gz uscan warning: In directory ., downloading ftp://ftp.ebi.edu.au/pub/software/MEME/index.html4.9.0/meme_4.9.0_4.tar.gz failed: 404 Can't chdir to index.html4.9.0 This is more or less expected, as the uscan(1) manpage says that the format with a separate downloads list page is HTTP-only. It's unexpected that it is able to report the latest version though. I can confirm that I have read the restriction for ftp - but considering that the version detection worked properly I failed to find a reason why regexp operations on strings should not work. But the problem is solved anyway. Thanks to Bart Martens for the proper solution. Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130214111540.gc12...@an3as.eu
Re: Watch file help needed
On Wed, Feb 13, 2013 at 04:39:11PM +0100, Andreas Tille wrote: the watch file of the prospective package meme[1] is reporting a new version but fails to download the latest version (4.9.0.4). It was working nicely with version 4.9.0.3. I just get: For the reference: opts=uversionmangle=s/_/./ \ ftp://ftp.ebi.edu.au/pub/software/MEME/index.html .*/meme_([\d\._]+)\.tar\.gz $ uscan --verbose --force-download ... 3.5.1/meme_3.5.1.tar.gz 3.5.0/meme_3.5.0.tar.gz Newest version on remote site is 4.9.0.4, local version is 4.9.0.3 = Forcing download as requested -- Downloading updated package meme_4.9.0_4.tar.gz uscan warning: In directory ., downloading ftp://ftp.ebi.edu.au/pub/software/MEME/index.html4.9.0/meme_4.9.0_4.tar.gz failed: 404 Can't chdir to index.html4.9.0 This is more or less expected, as the uscan(1) manpage says that the format with a separate downloads list page is HTTP-only. It's unexpected that it is able to report the latest version though. -- WBR, wRAR signature.asc Description: Digital signature
Re: Watch file help needed
On Wed, Feb 13, 2013 at 04:39:11PM +0100, Andreas Tille wrote: Hi, the watch file of the prospective package meme[1] is reporting a new [1] svn://svn.debian.org/debian-med/trunk/packages/meme/trunk version but fails to download the latest version (4.9.0.4). It was working nicely with version 4.9.0.3. I just get: $ uscan --verbose --force-download ... 3.5.1/meme_3.5.1.tar.gz 3.5.0/meme_3.5.0.tar.gz Newest version on remote site is 4.9.0.4, local version is 4.9.0.3 = Forcing download as requested -- Downloading updated package meme_4.9.0_4.tar.gz uscan warning: In directory ., downloading ftp://ftp.ebi.edu.au/pub/software/MEME/index.html4.9.0/meme_4.9.0_4.tar.gz failed: 404 Can't chdir to index.html4.9.0 -- Scan finished I tried some downloadurlmangle replacements with no effect at all. Any hint would be welcome I suggest to use this : version=3 opts=uversionmangle=s/_/./g \ http://ebi.edu.au/ftp/software/MEME/index.html .*/meme[_\-](\d.*)\.(?:tgz|tbz2|txz|tar\.(?:gz|bz2|xz)) Regards, Bart Martens -- To UNSUBSCRIBE, email to debian-mentors-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130213174019.gb8...@master.debian.org
