To all:
The following script will allow you to build mp4 videos from the jpgs
stored on the NOAA weather bureau website for the full disk GOES-16
jpgs. This script can be changed to accommodate the GOES-18 (west) or
the newer GOES-19 satellite images.
The bash script is expecting a frame count, since the GOES jps are taken
a certain few minutes apart, the full disk files are 5 mins apart, for
example, so the default is 576 frames or exactly 2 days time lapse.
The wget download takes awhile to accomplish, and during that time more
jpgs are added to the website, so the program turns around again and
grabs some more frames, and then the 3rd time grabs the new frames
again. 3 tries are sufficient to get up to date on all the jpgs.
There are different jpgs sizes available, 1808x1808 pixels, 5424x5424
pixels and the largest possible at 10848x10848 pixels. It is NOT
recommended to grab the last 2 sizes since the bash script will create
very large mp4 videos.
The script here is set to default to 1808x1808 pixels.
One important note: If you want to grab the mesoscale videos, those are
a completely different size and located on a different folder than the
default folder here, which is for just GOES-16 full disk jpgs.
It is not hard to change the script to grab the GOES-18 or GOES-19 full
disk videos. I might add this feature to the script file.
You will need to decide the root location to execute this file as it
creates a tmp folder and does all the work in the tmp folder then moves
the mp4 to the root location.
Having said all this (hopefully covering all the caveats) here is the
bash script which uses wget to grab the jpgs. This is more reliable than
grabbing the browser cache files.
-------------- make-GOES16-fullvid.sh -----------
tag=`date +"%Y-%m-%d %H:%M:%S.%N" | md5sum | cut -c1-32`
strt=`date +"%s"`
root="/home/owner/Desktop"
bld=$root/tmp
frames=$1
if [ -z $frames ] then
echo "Setting frames = 576 or 48 hr or 2 days"
frames=576
fi
echo "Selected $frames to display"
if [ ! -d $bld ] ; then
mkdir $bld
fi
## Cleanup temporary processing folder
cd $bld
ls * 2>/dev/null | sed "s/^/rm /" > tmp000.$tag 2>/dev/null
bash tmp000.$tag 2>/dev/null
## get the index.html.tmp file list
wget -A jpg /https://cdn.star.nesdis.noaa.gov/GOES16/GLM/FD/EXTENT3/
fld="https://cdn.star.nesdis.noaa.gov/GOES16/GLM/FD/EXTENT3/";
## sizes are 1808, 5424 and 10848
pix=1808
iv=0
echo "Grabbing $pix x $pix jpgs"
sleep 2
wget -A jpg $fld
mv index.html.tmp index.html.tmp.$iv
cat index.html.tmp.$iv | grep $pix | cut -f2 -d"\"" | grep -v ^$pix |
grep $pix > tmp000.$tag
cat tmp000.$tag | tail -576 | sed "s/^/wget -A jpg
https:\/\/cdn.star.nesdis.noaa.gov\/GOES16\/GLM\/FD\/EXTENT3\//" >
tmp001.$tag
bash tmp001.$tag
szz=`cat tmp001.$tag | wc -l`
echo "Grabbed $szz jpgs"
echo "checking for new jpgs"
wget -A jpg $fld
iv=`expr $iv + 1`
mv index.html.tmp index.html.tmp.$iv
cat index.html.tmp.$iv | grep $pix | cut -f2 -d"\"" | grep -v ^$pix |
grep $pix > tmp002.$tag
cat tmp000.$tag tmp002.$tag | sort -u > tmp001.$tag
cat tmp000.$tag tmp001.$tag | sort | uniq -c | grep -v " 2 " | sed
"s/ 1 //" > tmp002.$tag
## update tmp.000 index
cat tmp000.$tag tmp002.$tag > tmp001.$tag
mv tmp001.$tag tmp000.$tag
cat tmp002.$tag | sed "s/^/wget -A jpg
https:\/\/cdn.star.nesdis.noaa.gov\/GOES16\/GLM\/FD\/EXTENT3\//" >
tmp001.$tag
bash tmp001.$tag
szz=`cat tmp001.$tag | wc -l`
echo "Grabbed $szz new jpgs"
echo "checking for newer jpgs"
wget -A jpg $fld
iv=`expr $iv + 1`
mv index.html.tmp index.html.tmp.$iv
cat index.html.tmp.$iv | grep $pix | cut -f2 -d"\"" | grep -v ^$pix |
grep $pix > tmp002.$tag
cat tmp000.$tag tmp002.$tag | sort -u > tmp001.$tag
cat tmp000.$tag tmp001.$tag | sort | uniq -c | grep -v " 2 " | sed
"s/ 1 //" > tmp002.$tag
## update tmp.000 index
cat tmp000.$tag tmp002.$tag > tmp001.$tag
mv tmp001.$tag tmp000.$tag
cat tmp002.$tag | sed "s/^/wget -A jpg
https:\/\/cdn.star.nesdis.noaa.gov\/GOES16\/GLM\/FD\/EXTENT3\//" >
tmp001.$tag
bash tmp001.$tag
szz=`cat tmp001.$tag | wc -l`
echo "Grabbed $szz newer jpgs"
rm tmp001.$tag
## create mp4 file
ls *.jpg > tmp002.$tag
szz=`cat tmp002.$tag | wc -l`
nsz=`ls *jpg | wc -l`
echo "Found: $nsz jpgs"
fl=`ls *jpg | head -1`
lbl=`identify $fl | cut -c1-33 | sed "s/$/\.mp4/" `
sz=`cat tmp002.$tag | wc -l`
sz=`expr $sz + 10000`
seq 10001 $sz | sed "s/$/\.jpg/" | sed "s/^1//" > tmp003.$tag
paste tmp002.$tag tmp003.$tag | sed "s/^/mv /" | tr "\t" " " > tmp004.$tag
bash tmp004.$tag
ffmpeg -framerate 10 -i %4d.jpg -c:v libx264 -r 5 $lbl > /dev/null
2>/dev/null
cat tmp002.$tag | sed "s/^/rm /" > tmp005.$tag
bash tmp005.$tag 2>/dev/null
## cleanup and post stats and mp4 file name and processing time
rm *.jpg
rm index.html.tmp*
rm tmp*.$tag
echo "Found $szz frames:"
echo "final: $lbl"
stp=`date +"%s"`
elap=`echo "$stp - $strt" | gp -q`
date -d@$elap -u +%H:%M:%S | tr ":" "z" | sed "s/^/hr:/" | sed "s/z/
min:/" | sed "s/z/ secs:/" | sed "s/^/Elapsed time: /"
mv $bld/*mp4 $root
nm=`ls $root/*mp4`
echo "Video: $nm"
exit
