Re: OT help with wget
Alternatively, using Google's Takeout service will include images etc. See things like https://www.peggyktc.com/2021/08/back-up-all-your-blogger-account-data.html for details. On Thu, Feb 17, 2022 at 4:08 PM MacFH - C E Macfarlane - News wrote: > > I've sometimes used this bash script, which originally I wrote to > download a section of a site concerning some hardware I owned when the > site looked like it was about to close down, which in fact it did, so in > due course I was glad that I'd had the foresight. > > I usually find that it gets related images, etc, except where they > aren't stored in the same sub tree as the root document. > > (Note, beware unintended line wrap) > > wGetList.sh > === > > #!/bin/sh > # The name of the directory containing this script > DIRY="${0%/*}/" > # echo "Directory: ${DIRY}" > > # The filename of this script > SCRIPT="${0##*/}" > # echo "Filename: ${SCRIPT}" > > # User Agent string - Chrome > UAGENT="--user-agent='Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) > Gecko/20100101 Firefox/56.0'" > > # Options for wget command > OPTIONS="-F -r -c -nc -t 3 -T 60 --retry-connrefused -w 20 --random-wait > --waitretry=60 --no-check-certificate ${UAGENT}" > > # Input list of URLs to retrieve > LIST="" > LOG="" > if [ "${1:0:4}" != "http" ] > then > if [ -f "${1}" ] > then > LIST="-i ${1}" > LOG="${1%.*}.log" > shift > OPTIONS="${OPTIONS} ${LIST} ${*}" > else > echo "WARNING - url list file '$1' not found!" > exit 1 > fi > else > OPTIONS="${OPTIONS} ${*}" > LOG="${1##*//}" > LOG="${LOG%%/*}.log" > fi > > > # WGET the files in the list of URLs > echo "echo ${OPTIONS} | xargs wget > \"${LOG}\" 2>&1" > echo ${OPTIONS} | xargs wget > "${LOG}" 2>&1 & > > > On 17/02/2022 17:03, Dave Widgery wrote: > > Hi > > Sorry I know this is very OT but i thought there might be a few people > > here that might be able to help and possibly email me directly. > > We have several blogs (using googles blogger) that my wife has created > > over the years, but I want to create local copy's on my PC. I used the > > following command. > > wget --mirror --convert-links --adjust-extension --page-requisites > > --no-parent http://.blogspot.com > > and it created a full structure of the blog on my PC but it still > > relies on links to external websites for the images, can anybody > > suggest how to get it to also download copy's of the all the images as > > well? > > Again sorry for the OT post but I have been going round in circles for a > > while. > > Dave > > > > ___ > > get_iplayer mailing list > > get_iplayer@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/get_iplayer > > > ___ > get_iplayer mailing list > get_iplayer@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/get_iplayer ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: OT help with wget
I've sometimes used this bash script, which originally I wrote to download a section of a site concerning some hardware I owned when the site looked like it was about to close down, which in fact it did, so in due course I was glad that I'd had the foresight. I usually find that it gets related images, etc, except where they aren't stored in the same sub tree as the root document. (Note, beware unintended line wrap) wGetList.sh === #!/bin/sh # The name of the directory containing this script DIRY="${0%/*}/" # echo "Directory: ${DIRY}" # The filename of this script SCRIPT="${0##*/}" # echo "Filename: ${SCRIPT}" # User Agent string - Chrome UAGENT="--user-agent='Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0'" # Options for wget command OPTIONS="-F -r -c -nc -t 3 -T 60 --retry-connrefused -w 20 --random-wait --waitretry=60 --no-check-certificate ${UAGENT}" # Input list of URLs to retrieve LIST="" LOG="" if [ "${1:0:4}" != "http" ] then if [ -f "${1}" ] then LIST="-i ${1}" LOG="${1%.*}.log" shift OPTIONS="${OPTIONS} ${LIST} ${*}" else echo "WARNING - url list file '$1' not found!" exit 1 fi else OPTIONS="${OPTIONS} ${*}" LOG="${1##*//}" LOG="${LOG%%/*}.log" fi # WGET the files in the list of URLs echo "echo ${OPTIONS} | xargs wget > \"${LOG}\" 2>&1" echo ${OPTIONS} | xargs wget > "${LOG}" 2>&1 & On 17/02/2022 17:03, Dave Widgery wrote: Hi Sorry I know this is very OT but i thought there might be a few people here that might be able to help and possibly email me directly. We have several blogs (using googles blogger) that my wife has created over the years, but I want to create local copy's on my PC. I used the following command. wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://.blogspot.com and it created a full structure of the blog on my PC but it still relies on links to external websites for the images, can anybody suggest how to get it to also download copy's of the all the images as well? Again sorry for the OT post but I have been going round in circles for a while. Dave ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
OT help with wget
Hi Sorry I know this is very OT but i thought there might be a few people here that might be able to help and possibly email me directly. We have several blogs (using googles blogger) that my wife has created over the years, but I want to create local copy's on my PC. I used the following command. wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://.blogspot.com and it created a full structure of the blog on my PC but it still relies on links to external websites for the images, can anybody suggest how to get it to also download copy's of the all the images as well? Again sorry for the OT post but I have been going round in circles for a while. Dave ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer