Fifty frames per second with half of them redundant.
Hi, I'm used to an hour of iPlayer video needing about 1 GiB. In the past, this doubled for a while because the frame rate doubled from 25 to 50 per second. But stepping through the frames, say with mpv(1)'s ‘.’, showed the first frame of a pair will be a scene update and the second is a very minor adjustment of the pixels. So the camera shot might move in frames 1, 3, 5, ... and nothing much happen in frames 2, 4, 6... A waste for the BBC and me. Then downloads went back to the normal 1 GiB/hour. Recently, like the last few weeks, it's doubled again with the same cause. Take PID m001x0zq. ffprobe(1) shows Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 5058 kb/s, 50 fps, 50 tbr, 90k tbn, 100 tbc (default) Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default) Stream #0:2: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 192x108 [SAR 72:72 DAR 16:9], 90k tbr, 90k tbn, 90k tbc Does anyone have insights as to why this happens and what causes the BBC to return to normal? Is there anything I can do to force a sane 25 fps without dropping quality? -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Subtitles Unusually Not Available.
Hi, > > As I said, I've made the effort to tell the BBC of the problem. > > They have already emailed me a ‘case number’ from their ticketing > > system. Just an update... the BBC say this is now fixed and it appears so. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Subtitles Unusually Not Available.
Hi David, > > I've now finished spending fifteen minutes finding a means of > > filling in a ‘Contact Us [if you're a masochist]’ form, lying about > > the mandatory phone number, verifying the email address, etc. > > So you won't get an e-mail reply, and you will need to keep polling > the Web site. I'm unclear how what I wrote can be misinterpreted, but I'm happy to be told. :-) As I said, I've made the effort to tell the BBC of the problem. They have already emailed me a ‘case number’ from their ticketing system. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Subtitles Unusually Not Available.
Hi Mark, > > It's probably a ‘Beeb problem’, as you say. > > In some of these cases of technical iPlayer slips/omissions, > e-mailing the BBC gets them fixed fairly promptly. I did try emailing an old address I had for them but it auto-replied with a ‘We don't listen here any more’. I've now finished spending fifteen minutes finding a means of filling in a ‘Contact Us [if you're a masochist]’ form, lying about the mandatory phone number, verifying the email address, etc. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Subtitles Unusually Not Available.
Hi Chris, > INFO: Skipping 'iplayer' version > > It would seem to be a Beeb problem and not something you're doing, or > not doing ;-) Thanks. I notice that https://www.bbc.co.uk/iplayer/episode/m001b8js/beechgrove-2022-episode-20 says ‘We have removed the feature on the spindle bush which can be harmful to dogs if consumed in quantity. We do not advise planting in an area that is accessed by dogs who are unsupervised.’ Perhaps the editing of the programme means the normal ‘iplayer’ version of the subtitles, which is the only remaining version in the default list for which a check is done, isn't available but some other version is, one which isn't in get_iplayer's list of versions. Though if I play the programme on the iPlayer site, no subtitles appear and I don't see a way to turn them on but then playing in the browser is not normally something I do. It's probably a ‘Beeb problem’, as you say. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Subtitles Unusually Not Available.
Hi, Can anyone successfully obtain the subtitles for PID m001b8js? They're normally available for that series but this one episode lacks them AFAICS and I want to check it's not something I'm doing wrong. ... INFO: No streams available for 'signed' version (m001bgtm) - skipping INFO: No media streams found for 'original' version (m001b8jr) - deleting INFO: No media streams found for 'signed' version (m001bgtm) - deleting INFO: Processing tv: 'Beechgrove: 2022 - 20. Episode 20 (m001b8js)' INFO: Searching for versions: iplayer INFO: Subtitles not available for version(s): iplayer -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Curating "In Our Time" (IOT) downloads.
Hi Budge, > > Some of the ‘Unsorted’ ones have a PID and ‘./get_iplayer -i --pid > > b075t5mn’ shows > > > > categories: Factual,History,Discussion & Talk > > category:Factual ... > where did you get the categories line above? It's in the output of running the get_iplayer program with the arguments I showed above. But another reply has pointed out the same detail is probably in each of the MP3 files, which is easier to access. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Curating "In Our Time" (IOT) downloads.
Hi Budge, > file:///home/alastair/NFS_Multimedia_NFS/AV_multimedia/Music/Radio_Programme/In_Our_Time_Science/153 > > In_Our_Time_Archive_Science_-_IOT_The_Royal_Society_and_British_Science_Episode_4_iots_20100107-0900a.mp3 > file:///home/alastair/NFS_Multimedia_NFS/AV_multimedia/Music/Radio_Programme/In_Our_Time_Unsorted/In_Our_Time_-_716._The_Sikh_Empire_b075t5mn_default.m4a > file:///home/alastair/NFS_Multimedia_NFS/AV_multimedia/Music/Radio_Programme/In_Our_Time_Unsorted/In_Our_Time_-_A_Midsummer_Nights_Dream_m00046rp_podcast.m4a > file:///home/alastair/NFS_Multimedia_NFS/AV_multimedia/Music/Radio_Programme/In_Our_Time_Unsorted/In_Our_Time_With_Melvyn_Bragg_-_IOT_Zen_04_Dec_14_iot_20141204-1140a.mp3 Some of the ‘Unsorted’ ones have a PID and ‘./get_iplayer -i --pid b075t5mn’ shows categories: Factual,History,Discussion & Talk category:Factual Is ‘Factual’ the kind of thing which would replace ‘Unsorted’? -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Curating "In Our Time" (IOT) downloads.
Hi Budge, > The categories were Culture, History, Philosophy, Religion and > Science. This seems to have stopped around 2012, possible due to BBC > format changes and since then they have all been saved in my system as > "Unsorted" and for a while these were also numbered but are no longer, > possibly due to changes in my own GiP setup over the years. Please show the list some example filenames, both those old ones which are in their correct category and some new ‘unsorted’ ones. This will tell us what information can be gleaned from them, e.g. an iPlayer PID. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Every download command results in > being displayed!
Hi Sharon, > then issue the following command 'get_iplayer -g 33840' & onwards, but > no matter what I do the end of the list shows up as '>' and doesn't > allow me to download anything at all! My hunch... One of the parameters you have entered has a single quote, ‘'’, and that starts a quoted string which can cross the end of the first line into subsequent lines. The normal shell's prompt of ‘$’ changes to ‘>’ to show this line is a continuation of the previous one. $ echo foo foo $ echo foo'bar > xyzzy' foobar xyzzy $ Remove the quoting behaviour of the single quote by prefixing it with a backslash, ‘\’, which is an ‘escape hatch’ for proving a literal quote despite its special meaning. This is known as ‘escaping’, e.g. ‘escape the ' with a \’. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Off-topic - Paddington
Hi SB, > > IIRC, it's a little after thirty minutes into > > https://www.bbc.co.uk/programmes/p0bk5pd7 because it kicks off the > > start of the party after half an hour of waffle and re-cap. > > The Concert has gone from iPlayer if it was ever there. It certainly was there; I downloaded it using that PID. > 6777: The Jubilee Pudding: 70 Years in the Baking - -, BBC One, m00178bg > 6925: The Queens Platinum Jubilee - Trooping the Colour, BBC News, > m0017xfk > 6926: The Queens Platinum Jubilee - Platinum Beacons: Lighting up the > Jubilee, BBC News, m0017xgb > 6927: The Queens Platinum Jubilee - A Service of Thanksgiving, BBC > One, m0017xjp > 6928: The Queens Platinum Jubilee - The Platinum Pageant, BBC One, > m00183k3 > 6929: The Queens Platinum Jubilee - What a Weekend!, BBC Two, m001847z > 6930: The Queens Platinum Jubilee - Platinum Party at the Palace, BBC > One, p0bk5pd7 > INFO: 8 matching programmes Did you notice that PID at 6930? See the end of the line. If you open the URL I gave then you'll see that is the programme and Paddington is in it. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Off-topic - Paddington
Hi David, > Off-topic, but has any a full resolution link to the full Queen Meets > Paddington" video? Can't find it on the iPlayer IIRC, it's a little after thirty minutes into https://www.bbc.co.uk/programmes/p0bk5pd7 because it kicks off the start of the party after half an hour of waffle and re-cap. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Audio portion of dash downloads slow
Hi, Jeremy wrote: > > INFO: Downloaded: 158.00 MB (02:42:25) [2538] in 00:31:30 @ 0.67 Mb/s units(1) is handy for that kind of thing. $ units -1v '158 MiB / (31 min + 30 s)' Mibit/s 158 MiB / (31 min + 30 s) = 0.66878307 Mibit/s -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: v2 - get_iplayer Package Issue For Debian / Raspbian on openSUSE Build Service
Hi Charles, > > sudo rm -v /etc/apt/trusted.gpg.d/home:m-grant-prg.gpg > > You'll need to escape the ":" in the old keyring filename The original is correct. ‘:’ in filenames is not significant to the Bourne shell and its descendants. $ touch :foo b:a:r xyzzy: $ ls b:a:r :foo xyzzy: $ > obviously tab completion will do this for you It doesn't need to for most things and it's annoying it does. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Repair Shop - series 5 episodes 1 - 20
Hi CJB, > Wondering if anyone downloaded these episodes? Actually we are seeking > the subtitle files please. > > Note: episodes 11 - 40 were never aired even if the iPlayer website says so!! I have these. 1 The_Repair_Shop_Series_5_-_01._Episode_1_m000bhh1_original.srt 2 The_Repair_Shop_Series_5_-_02._Episode_2_m000bh2z_original.srt 3 The_Repair_Shop_Series_5_-_03._Episode_3_m000bh8b_original.srt 4 The_Repair_Shop_Series_5_-_04._Episode_4_m000bh75_editorial.srt 5 The_Repair_Shop_Series_5_-_05._Episode_5_m000bj1n_original.srt 6 The_Repair_Shop_Series_5_-_06._Episode_6_m000bq1w_original.srt 7 The_Repair_Shop_Series_5_-_07._Episode_7_m000bpqf_editorial.srt 8 The_Repair_Shop_Series_5_-_08._Episode_8_m000bpzt_original.srt 9 The_Repair_Shop_Series_5_-_09._Episode_9_m000bq38_original.srt 10 The_Repair_Shop_Series_5_-_10._Episode_10_m000bqm3_original.srt 11 The_Repair_Shop_Series_5_-_11._Episode_11_m000by2r_original.srt 12 The_Repair_Shop_Series_5_-_12._Episode_12_m000by41_original.srt 13 The_Repair_Shop_Series_5_-_13._Episode_13_m000by4w_original.srt 14 The_Repair_Shop_Series_5_-_14._Episode_14_m000by7m_original.srt 15 The_Repair_Shop_Series_5_-_15._Episode_15_m000bytq_original.srt 16 The_Repair_Shop_Series_5_-_16._Episode_16_m000c616_original.srt 17 The_Repair_Shop_Series_5_-_17._Episode_17_m000c62j_original.srt 18 The_Repair_Shop_Series_5_-_18._Episode_18_m000c64m_original.srt 19 The_Repair_Shop_Series_5_-_19._Episode_19_m000c63n_original.srt 20 The_Repair_Shop_Series_5_-_20._Episode_20_m000c6mh_original.srt Email me off-list if they're still of use to you. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: OT BBC Iplayer on firestick problems
Hi Dave, > I have about 15Mbs download on a good day and if nothing else is going > on, I regularly get 8-10Mbs while doing a get_iplayer download and > that is through the same proxy so it sounds like it is probably the > app causing the problem. Perhaps the two are ending up on different CDNs? -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Slow speed
Hi, RS wrote: > Your only real remedy is to complain, complain and complain again to > your ISP, or find a better ISP. I'd have thought it's the CDN choice that's being made rather than ISP. For example, having the line ‘exclude-supplier bidi’ in my ~/.get_iplayer/options fixed a problem for me a few months back. The OP needs to investigate what CDN/supplier is being used when he's having problems. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Re: Search Fails to Find Early ‘Last Tango in Halifax’.
Hi James, > It's my understanding that GiP indexes TV programs via the schedules. Oh, I see your point. > The availability of programs on the iPlayer no longer has a one-to-one > mapping with what will be or has been on TV. If the GiP search isn't > going to catch up to that fact, it may as well just be removed. No, it's too useful to be removed. I see a tweet that makes me think yesterday's ‘Daily Politics’ with Andrew Neil might be worth a watch and search lets me find its index to add to the PVR without leaving the terminal. It just needs to be defined well, and perhaps it is; I haven't read its documentation in the years since I first used it. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Search Fails to Find Early ‘Last Tango in Halifax’.
Hi, I recently noticed S05E01 of ‘Last Tango in Halifax’ had been added to the available programs. It's also returned by a search. $ ./get_iplayer --nopurge -e 31536000 --future tango get_iplayer v3.22, Copyright (C) 2008-2010 Phil Lewis This program comes with ABSOLUTELY NO WARRANTY; for details use --warranty. This is free software, and you are welcome to redistribute it under certain conditions; use --conditions for details. Matches: 5621: Last Tango in Halifax: Series 5 - Episode 1, BBC One, m000fs1z INFO: 1 matching programmes $ Note the old version, v3.22. Perhaps that's related to my problem. Poking about the iPlayer web site, I see many of the previous episodes are available, e.g. S01E01 at https://www.bbc.co.uk/programmes/b01p1q71 says ‘Watch now’. Why isn't it found by the search above? ‘./get_iplayer --no-purge --future -e 31536000 -i --pid b01p1q71’ finds it and also suggests it's available for a while. expires: in 348 days 22 hours (2021-01-30T12:00:00+00:00) When I refresh, I do it twice, the first time with ‘--refresh --refresh-limit-tv=30’ and the second with ‘--refresh --refresh-future’. I'm happy to update my get_iplayer and re-apply/re-edit my patches, but would like to hear that the search returns copious episodes for another first before bothering. And if it doesn't, does anyone here know why not? -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Available: Yes, Prime Minister, The Tangled Web.
Hi, I've deleted the emails from the earlier thread discussing PID b037tb14 being unavailable. I agree, it was then. In another periodic run of my PVR queue, it downloaded. Duration: 00:28:15.04, start: 0.00, bitrate: 247 kb/s Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt470bg), 960x540 [SAR 1:1 DAR 16:9], 1597 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default) Stream #0:1(eng): Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 96 kb/s (default) https://www.bbc.co.uk/programmes/b037tb14 matches this with a ‘Watch now’. The thread starts with MacFH at http://lists.infradead.org/pipermail/get_iplayer/2020-February/012683.html -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Call the Midwife Chistmas 2019
Hi terry, > I am attempting to download m000csm5 and I am having odd results. > Basically, it starts out downloading DASH audio and then restarts > downloading audio+video. twice it finished downloading and failed in > ffmpeg. Unable to convert. > > Has anyone successfully downloaded this program? What mode did you > use? Not specifying the mode worked for me, but only after it had tried and failed with the bidi CDN. INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (dvfhd1/bi) [audio] WARNING: Failed to download file segment [0] It then moved onto Limelight, and that worked. INFO: Downloaded: 86.06 MB (01:28:30) @ 57.37 Mb/s (dvfhd2/ll) [audio] INFO: Downloaded: 3226.23 MB (01:28:30) @ 158.34 Mb/s (dvfhd2/ll) [video] INFO: Converting to MPEG-TS INFO: Converting to MP4 INFO: Tagging MP4 So you may wish to try adding ‘--exclude-supplier bidi’. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: His Dark Materials m000csdk: only audiodescribed available?
Hi Don, > No, Ralph, I didn't request the AD version, but that's what I got by > default. > > From what I can see on the iPlayer website, only audiodescribed > versions are available. Playing using VLC doesn't play the AD. VLC has no choice but to play the single audio stream in the MP4 file. Whether technical or audiodescribed, there is only one audio stream present. $ ffprobe -i His_Dark_Materials_Series_1_-_08._Betrayal_m000csdk_technical.mp4 |& > grep Audio Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default) $ ffprobe -i His_Dark_Materials_Series_1_-_08._Betrayal_m000csdk_audiodescribed.mp4 |& > grep Audio Stream #0:1(eng): Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 96 kb/s (default) So I don't know how VLC is avoiding reading out the audio descriptions unless it's not actually an audiodescribed download that was obtained. get_iplayer could obtain both audio streams and put them in one MP4 file along with the video, and VLC and other players could then present the choice of audiodescribed or not, just as foreign-language dubbing is sometimes present in non-iPlayer material. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: His Dark Materials m000csdk: only audiodescribed available?
Hi Richard,
> > Oh, download_history, good idea. I add each manually to the PVR.
> > Here's my episodenum, mode, versions, and the end of the filename.
> >
> > $ awk -F\| '$2 ~ /^His Dark/ {sub(/.*_/, "", $7); print $(NF-2), $6,
> > $8, $7}' \
> > > ~/.get_iplayer/download_history
> > 1 hvfhd1 audiodescribed,original original.mp4
> > 2 hvfhd1 audiodescribed,technical technical.mp4
> > 3 hvfhd1 audiodescribed,technical technical.mp4
> > 4 hvfhd1 audiodescribed,technical technical.mp4
> > 5 dvfhd2 audiodescribed,original original.mp4
> > 6 dvfhd2 audiodescribed,technical technical.mp4
> > 7 dvfhd2 audiodescribed,original original.mp4
> > 8 dvfhd1 audiodescribed,technical technical.mp4
> > $
>
> The versions field is telling you the available versions, and the end
> of the filename is telling you the version you downloaded.
Yes, that's why I selected both of them. :-)
> I can't see anything in what you have shown which suggests you got the
> audiodescribed version when you didn't want it.
Because my email above was after the one where I thanked you for
pointing out CDN-selection might help:
http://lists.infradead.org/pipermail/get_iplayer/2019-December/012652.html
Thus I managed to obtain technical by the time I analysed
download_history.
> download_history is written incrementally (append mode), so if you
> have downloaded the last episode twice there ought to be two records.
I tend to ed(1) download_history to delete a PID if I want to try again
rather than use --force as I'm not certain of every aspect of the
latter's actions. I've just obtained E08 again without excluding the
Bidi CDN,
INFO: Downloaded: 41.70 MB (00:57:20) @ 17.56 Mb/s (dvfxsd2/ll) [audio]
INFO: Downloaded: 614.02 MB (00:57:20) @ 68.22 Mb/s (dvfxsd2/ll) [video]
and now get two entries for it, as you suggest.
$ awk -F\| '$2 ~ /^His Dark/ {sub(/.*_/, "", $7); print $(NF-2), $6, $8,
$7}' \
> ~/.get_iplayer/download_history
...
8 dvfhd1 audiodescribed,technical technical.mp4
8 dvfxsd2 audiodescribed,technical audiodescribed.mp4
$
Interesting how I once again get the low-resolution audiodescribed
version despite those ‘Downloaded’ showing the ‘/ll’ CDN. If I download
it a third time, adding ‘--exclude-supplier bidi’ then I upgrade again
to technical, still /ll.
INFO: Downloaded: 55.78 MB (00:57:20) @ 63.75 Mb/s (dvfhd1/ll) [audio]
INFO: Downloaded: 1953.86 MB (00:57:20) @ 156.31 Mb/s (dvfhd1/ll) [video]
I think what happens is without the bidi exclusion, technical is
attempted from bidi, that fails, audiodescribed is tried instead, that
also fails from bidi, and then the next CDN is tried, /ll, that works,
but we haven't started at the beginning of the versions, instead staying
stuck on audiodescribed:
INFO: Downloading tv: 'His Dark Materials: Series 1 - 08. Betrayal
(m000csdk) [technical]'
INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (dvfhd1/bi) [audio]
WARNING: Failed to download file segment [0]
WARNING: Response: 400 Bad Request
WARNING: No streams available for 'technical' version (m000d182) - skipping
(retry)
INFO: Downloading tv: 'His Dark Materials: Series 1 - 08. Betrayal
(m000csdk) [audiodescribed]'
INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (dvfxsd1/bi) [audio]
WARNING: Failed to download file segment [0]
WARNING: Response: 400 Bad Request
INFO: Downloaded: 41.70 MB (00:57:20) @ 17.56 Mb/s (dvfxsd2/ll) [audio]
INFO: Downloaded: 614.02 MB (00:57:20) @ 68.22 Mb/s (dvfxsd2/ll) [video]
--
Cheers, Ralph.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: His Dark Materials m000csdk: only audiodescribed available?
Hi Don,
> I've got episode 1 as hvfhd2, the rest as hvfhd1
> Episodes 1, 5 and 7 are original, the rest technical.
Oh, download_history, good idea. I add each manually to the PVR.
Here's my episodenum, mode, versions, and the end of the filename.
$ awk -F\| '$2 ~ /^His Dark/ {sub(/.*_/, "", $7); print $(NF-2), $6, $8,
$7}' \
> ~/.get_iplayer/download_history
1 hvfhd1 audiodescribed,original original.mp4
2 hvfhd1 audiodescribed,technical technical.mp4
3 hvfhd1 audiodescribed,technical technical.mp4
4 hvfhd1 audiodescribed,technical technical.mp4
5 dvfhd2 audiodescribed,original original.mp4
6 dvfhd2 audiodescribed,technical technical.mp4
7 dvfhd2 audiodescribed,original original.mp4
8 dvfhd1 audiodescribed,technical technical.mp4
$
> All are audiodescribed.
As in a voice describes the picture? Because that's what you wanted?
--
Cheers, Ralph.
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: His Dark Materials m000csdk: only audiodescribed available?
Hi Richard, > > Specifying ‘--modes dvfhd1,dvfhd2,dvfhd3’ picks the technical version. > > > > INFO: Downloading tv: 'His Dark Materials: Series 1 - 08. Betrayal > > (m000csdk) [technical]' > > > > INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (dvfhd1/bi) [audio] > > WARNING: Failed to download file segment [0] > > WARNING: Response: 400 Bad Request > > WARNING: No streams available for 'technical' version (m000d182) - > > skipping (retry) > > Have you tried a different CDN as your first choice? No, thanks, that didn't occur to me. Adding ‘--exclude-supplier bidi’ meant the download went well: technical, not audiodescribed, decent resolution. Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 4542 kb/s, 50 fps, 50 tbr, 90k tbn, 100 tbc (default) > I don't know why get_iplayer has not retried with the second and third > CDNSs you have specified. Me neither. It's not something I normally have to do. Another download straight after the above one that worked seemed to also have ‘bi’ problems and happily moved onto ‘ll’. INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (dvfhd1/bi) [audio] WARNING: Failed to download file segment [0] WARNING: Response: 400 Bad Request INFO: Downloaded: 42.57 MB (00:43:46) @ 56.75 Mb/s (dvfhd2/ll) [audio] Going back to --streaminfo on m000csdk, the non-bidi streams are listed. stream:dvfhd1 priority: 30 streamurl: https://mm.bidi.bbc.co.uk/vod-dash-uk/usp/auth/vod/piff_abr_full_hd/6daf0e-m000d8kd/vf_m000d8kd_4db45b44-d2c8-476c-a56e-82c3dc7bf180.ism.hlsv2.ism/dash/vf_m000d8kd_4db45b44-d2c8-476c-a56e-82c3dc7bf180.ism.hlsv2-video=507.dash?at=CvusXAqfab332c59c6ec9d0c6a5e1f062b8f92c81b20e9ec59a73dad71ac0 stream:dvfhd2 priority: 20 streamurl: https://vod-dash-uk-live.bbcfmt.hs.llnwd.net/usp/auth/vod/piff_abr_full_hd/6daf0e-m000d8kd/vf_m000d8kd_4db45b44-d2c8-476c-a56e-82c3dc7bf180.ism.hlsv2.ism/dash/vf_m000d8kd_4db45b44-d2c8-476c-a56e-82c3dc7bf180.ism.hlsv2-video=507.dash?s=1577175667=1577218867=6aad83bd5a305853f316983306c260c1 stream:dvfhd3 priority: 10 streamurl: https://vod-dash-uk-live.akamaized.net/usp/auth/vod/piff_abr_full_hd/6daf0e-m000d8kd/vf_m000d8kd_4db45b44-d2c8-476c-a56e-82c3dc7bf180.ism.hlsv2.ism/dash/vf_m000d8kd_4db45b44-d2c8-476c-a56e-82c3dc7bf180.ism.hlsv2-video=507.dash?__gda__=1577218867_a6308fb3a2add523aba9809ec4aec7cd I'm using version 3.22, but I don't see anything relevant at https://github.com/get-iplayer/get_iplayer/wiki/release320to329#changes-in-323 that suggests 3.23 would fare better. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: His Dark Materials m000csdk: only audiodescribed available?
Hi Mark, > If I don't see it under > https://www.bbc.co.uk/iplayer/help/known-issues Thanks, just checked, not there. > then I mention it on > https://www.bbc.co.uk/iplayer/help/questions/need-more-help/report-prog-problem Filled that it. Thanks for pointing them out. -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
His Dark Materials m000csdk: only audiodescribed available?
Hi, Am I understanding get_iplayer correctly? I've been successfully downloading episodes 1-7 of their adaption of ‘His Dark Materials’. Come the last episode, it's only available in low resolution and audiodescribed to boot meaning that voice-over from Big Brother is moonlighting with ‘Lyra is now entering the charnal house’. --info for the PID m000csdk shows web:https://www.bbc.co.uk/programmes/m000csdk Viewing that in Firefox shows audiodescribed and doesn't seem to let me turn it off. There's two versions. version:technical versions: audiodescribed,technical modes: audiodescribed: dvfxsd1,dvfxsd2,dvfxsd3,dvflow1,dvflow2,dvflow3 modesizes: audiodescribed: dvfxsd1=690MB,dvfxsd2=690MB,dvfxsd3=690MB, dvflow1=188MB,dvflow2=188MB,dvflow3=188MB [estimated sizes only] verpids:audiodescribed: p07ym92s modes: technical: dvfhd1,dvfhd2,dvfhd3,dvfsd1,dvfsd2,dvfsd3,dvfxsd1, dvfxsd2,dvfxsd3,dvfhigh1,dvfhigh2,dvfhigh3,dvfxhigh1,dvfxhigh2, dvfxhigh3,dvflow1,dvflow2,dvflow3,subtitles1,subtitles2 modesizes: technical: dvfhd1=2179MB,dvfhd2=2179MB,dvfhd3=2179MB,dvfsd1=1209MB, dvfsd2=1209MB,dvfsd3=1209MB,dvfxsd1=690MB,dvfxsd2=690MB, dvfxsd3=690MB,dvfhigh1=675MB,dvfhigh2=675MB,dvfhigh3=675MB, dvfxhigh1=356MB,dvfxhigh2=356MB,dvfxhigh3=356MB,dvflow1=188MB, dvflow2=188MB,dvflow3=188MB [estimated sizes only] verpids:technical: m000d182 Despite that, it seems no technical streams work. --streaminfo's first stream is stream:dvfhd1 audio_bitrate: 128 bitrate: 5070 expires: 2020-06-21T21:00:00Z ext: mp4 kind: video priority: 30 size: 2179466250 streamer: dash streamurl: https://mm.bidi.bbc.co.uk/vod-dash-uk/usp/auth/vod/piff_abr_full_hd/6daf0e-m000d8kd/vf_m000d8kd_4db45b44-d2c8-476c-a56e-82c3dc7bf180.ism.hlsv2.ism/dash/vf_m000d8kd_4db45b44-d2c8-476c-a56e-82c3dc7bf180.ism.hlsv2-video=507.dash?at=MAHcbNcH281380ce507a736d5d39479c6bc64bb2ae22018d59a710c115e00 type: gip_dvf_5070 dash h264 1280x720 50fps 5070kbps 128kbps mf_bidi/30 video_bitrate: 5070 Specifying ‘--modes dvfhd1,dvfhd2,dvfhd3’ picks the technical version. INFO: Downloading tv: 'His Dark Materials: Series 1 - 08. Betrayal (m000csdk) [technical]' INFO: Downloaded: 0.00 MB (00:00:00) @ 0.00 Mb/s (dvfhd1/bi) [audio] WARNING: Failed to download file segment [0] WARNING: Response: 400 Bad Request WARNING: No streams available for 'technical' version (m000d182) - skipping (retry) It seems to me the BBC encoding of this final episode has gone wrong, leaving just the audiodescribed version available. That's fair enough, but what's surprising is that they haven't noticed this themselves from their monitoring and alerts. Or am I misinterpreting get_iplayer's output and a decent resolution, Big-Brother-less audio, download is possible? -- Cheers, Ralph. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: --url, -recursive, multiple --pids
Hi Clive, > > > First of all, removing the space between the individual pids has > > > worked. This is odd because in the past, when I use the five > > > digit number to download multiple programs I just leave a space > > > between them, eg: > > > > > > get_iplayer 12345 12346 12347 12348 12349 -g > > > > Those `five digit number' are indexes, not PIDs. The --pid option > > does not change the meaning of arguments from indexes to PIDs; it > > takes a single argument that is a comma-separated list of PIDs. > > My experience is that if I enter the index numbers with each of five > digits separated by a space then it works, always. Try it on something > short like Tweet of the Day: No, I don't need to try it. I keep agreeing with you! :-) And you didn't ask about index numbers, but about PIDs. Please re-read what I've written. You are mixing up an index and a PID, `10302' v. `b0bg2ctf'. They are not the same thing and get_iplayer does not take an argument and work out which it is. It has to be an index unless it is the argument following --pid when it can be a comma-separated list of PIDs. These are the equivalent forms, given my index values here. 10299 10300 10301 --pid b0bbbzr2,b0bcmpgb,b0bg2ctf -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: --url, -recursive, multiple --pids
Hi Clive, > > Are you sure? With a single `-'? > > This old get_iplayer, I haven't upgraded yet, has --pid-recursive. > > Perhaps that does what you're thinking of; I've never used it. > > you are correct, there should have been two hyphens in old GiP > however, in this version, using one or two hyphens on the url tells > me: "Unknown option: recursive" and a search of the help page fails to > find the word 'recursive'. I've downloaded 3.17 now. $ ./get_iplayer --long-help |& grep recursive --pid-recursive Record all related episodes if value of --pid is a series or brand PID. Requires --pid. --pid-recursive-list If value of --pid is a series or brand PID, list available episodes but do not download. Implies --pid-recursive. Requires --pid. $ > > > I appear to have followed the syntax in the help screen > > --pid ,,... > > > > Don't add spaces after the comma? It makes them separate words. > > First of all, removing the space between the individual pids has > worked. This is odd because in the past, when I use the five digit > number to download multiple programs I just leave a space between > them, eg: > > get_iplayer 12345 12346 12347 12348 12349 -g Those `five digit number' are indexes, not PIDs. The --pid option does not change the meaning of arguments from indexes to PIDs; it takes a single argument that is a comma-separated list of PIDs. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: --url, -recursive, multiple --pids
Hi Clive, > In the past I would have typed: > > get_iplayer --url="xxx" -recursive Are you sure? With a single `-'? This old get_iplayer, I haven't upgraded yet, has --pid-recursive. Perhaps that does what you're thinking of; I've never used it. > get_iplayer --pid p06hcf2k, p06hcfgy, p06gdf2p, p06gddz0, p06gdd2d, > p06gdg0q, p06gdfvq, p06gdfmn, p06gdfqp, p06gdfh8, p06gdj6r, p06gdjb9, > p06dqrzv ... > I appear to have followed the syntax in the help screen --pid ,,... Don't add spaces after the comma? It makes them separate words. $ prargv get_iplayer --pid foo, bar, xyzzy 0 '/home/ralph/bin/prargv' 1 'get_iplayer' 2 '--pid' 3 'foo,' 4 'bar,' 5 'xyzzy' $ prargv get_iplayer --pid foo,bar,xyzzy 0 '/home/ralph/bin/prargv' 1 'get_iplayer' 2 '--pid' 3 'foo,bar,xyzzy' $ -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Exclude/Include patterns
Hi Richard, > > `--refresh-include' takes a comma-separated list of case-insensitive > > regexps, from looking at the `channels_filtered' subroutine, so > > «--refresh-include '^BBC Radio 4$'» should cut out the `Extra'. > > (The regexp is quoted for Unix.) > > Are you saying the onus is on the user to insert ^ and $ in a regex to > suppress greedy matching? What's happening is not greedy matching, but yes, if the user wants the regexp to stop matching after the `4' then they need to state that in some way, e.g. `4$'. > In the case of a search argument the documentation requires a regex. Yes. > There is no similar requirement for --refresh-include. That's a documentation error. > My view is that if the programmer chooses to use a regex there it is > up to him or her to construct the regex. Well, that's not what the code currently does. It splits the parameter on commas and treats each resulting part as a regexp, combining them with `|' so any of them matching is an overall match. For example, `foo,bar xyzzy' becomes the regexp /(foo|bar xyzzy)/i. The trailing `i' indicates case insensitive. > My first attempt was --refresh-include radio4. Nothing was found. I > concluded I had to match the channel name exactly as "BBC Radio 4". The regexp /radio4/i doesn't match the string `Radio 4'. > I was therefore surprised to have "BBC Radio 4 Extra" matched as well. The regexp /BBC Radio 4/i does match the string `BBC Radio 4 Extra'. > A more important question is do you agree that in order for > --refresh-include to work it has to be accompanied by > --refresh-exclude-groups national,regional,local? No idea. The subroutine I mentioned isn't particularly clear given I don't know the data structures it's walking so I didn't pick up on that. (More effort could have gone into distinctive identifiers to avoid many of them seemingly similar.) > > There should be a `INFO: Will refresh channel...' verbose log > > message for each channel that makes it through. > > Taking your last point first, the verbose messages are not very > helpful because they are accompanied by a lot of messages about the > schedule pages. Curiously for --refresh-include "radio 4" I get > > INFO: Will refresh channel BBC Radio 4 > INFO: Will refresh channel BBC Radio 4 > INFO: Will refresh channel BBC Radio 4 Extra That seems pretty helpful. It's confirming the channels that are matching the comma-separate list of regexps. I assume the first is there twice because it's a channel that's in two different groups. Perhaps the group could also be stated in each line. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Exclude/Include patterns
Hi Richard, > get_iplayer --refresh-exclude-groups-radio national,regional,local > --refresh-include "BBC Radio 4" -f --type radio > > It added 1840 radio programmes to the cache. As far as I could see > from a visual inspection scrolling through the newly created > radio.cache file all the programmes were from Radio 4, Radio 4 Extra > and the World Service when joined with Radio 4. `--refresh-include' takes a comma-separated list of case-insensitive regexps, from looking at the `channels_filtered' subroutine, so «--refresh-include '^BBC Radio 4$'» should cut out the `Extra'. (The regexp is quoted for Unix.) There should be a `INFO: Will refresh channel...' verbose log message for each channel that makes it through. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Exclude/Include patterns
Hi Az, > and I tried to exclude everything and just include those like so: It's not obvious from looking at `channels_filtered' in the get_iplayer script that they combine like that. > -v --cache-rebuild --type=radio --refresh-exclude ".*" > --refresh-include "BBC Radio 3,BBC Radio 4,BBC Radio 4 Extra" Have you tried just using `--refresh-include' without excluding everything? -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Disagreement Over Stream Claimed and Result.
Hi SP, > It seems you have swapped hvfxsd1 with hvfxhigh1. Thanks. I formatted it assuming every `type:' and `stream:' were paired; they're not. Thus I was pairing entries from different adjacent records. Sorry for the noise. BTW, regarding 960x540 25fps being inadequate compared to the discontinued 1280x720 25fps, a good example is gardening programs with lots of green leaf in shot, e.g. trees. As the camera pans, the background `pulses' at about 1 Hz in its updates; there's a drag where it doesn't update, and then it all updates at one. If a small presenter walks about in the shot then his face is just skin coloured with no features until he stops moving and it fills in. Never saw that with the same TV series (plural) at 1280x720 25fps, approximately 1 GiB per hour. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Disagreement Over Stream Claimed and Result.
Hi, If I `--streaminfo --pid b0b9dzbx' then it includes this output, reformatted for clarity. fps kbps stream gip_dvf_iplayer_827 dash h264 704x396 25 827 mf_akamai_uk_dash/1 dvfxsd1 '' '' '' '''' '' mf_limelight_uk_dash/2 dvfxhigh2 gip_hvf_iplayer_1216 hls '' '''' 1216 mf_akamai_uk_hls/1 hvfxsd1 '''' '' '''' '' mf_bidi_uk_hls/3 hvfxhigh2 '''' '' '''' '' mf_limelight_uk_hls/2 hvfxhigh3 gip_dvf_iplayer_1604 dash h264 960x540 25 1604 mf_akamai_uk_dash/1 hvfhd1 '' '' '' '''' '' mf_limelight_uk_dash/2 dvfxsd2 gip_hvf_iplayer_2040 hls '' '''' 2040 mf_akamai_uk_hls/1 subtitles1 '''' '' '''' '' mf_bidi_uk_hls/3 hvfxsd2 '''' '' '''' '' mf_limelight_uk_hls/2 hvfxsd3 On downloading, output includes INFO: Downloading tv: 'This Week - 05/07/2018 (b0b9dzbx) [original]' INFO: Downloaded: 719.21 MB (00:48:21) @ 100.94 Mb/s (hvfxsd1/bi) [audio+video] The only `hvfxsd1' in the stream info is the third line, 704x396, but `ffmpeg -i' correctly says `960x540', Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 960x540 [SAR 1:1 DAR 16:9], 1586 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default) that means one of the last was retrieved, `hvfxsd2' or `hvfxsd3'. Presumably `hvfxsd2' because that's a `bidi' that matches the `bi' CDN in the claimed `hvfxsd1/bi'? Am I misunderstanding, or is get_player misleading when stating the stream? -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Jim, > > Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could > > `--output /tmp' and you should see its intermediate files, and the > > final file, only appear there. `--command' could then move that > > final file to the SSD, or run a conversion command that writes to > > the SSD and then removes the tmpfs input. > > the df command listed various 'locations'. These included > > tmpfs 815696 1352 814334 /run > none 4078460 0 4078460 /run/shm > none 102400 8 102392 /run/usr /run/shm is half your RAM, as I'd expect. I'd also expect /tmp to be there. What does `df -T /tmp' show? Perhaps that's using SSD for all temporary files and you might not want that given your concerns. > Inside /run/usr I did find a directory for my user id and could use > that as the user. But it's deliberately constrained to be small. > However I don't know what the /run/shm is for, so need to do some > finding out... POSIX's communication methods of shared memory and semaphores; shm_overview(7), and sem_overview(7). Again, not what you want. > I did set up an explict ramdisc on another machine ages ago by adding > a line to the /etc/fstab file. But that fixed the size to 256 MB, > which in this context is tiny. System monitor says I have 8GB of ram. That's the right idea. Here, the system has /tmp be a tmpfs that is half the RAM, like the `shm' one above. They share one single half. `systemctl status tmp.mount' is what creates it on some systems these days, not an /etc/fstab entry. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Jim, > My concern is that I only have a total of 8GB of ram at present on the > machine, and the 1280x720 50fps files tend to come in at 2GB or more > per hour. Yes, you'd need to finish one download off, clearing RAM disk, before starting the next. > To save time and avoid running past 9am (and entering metered time) it > would be quickest to do all the fetches first. I too have metered broadband, but have automated use of it begin at 00:03 using timeout(1) to ensure it is killed by my 08:00 cut-off time if it didn't complete for some reason. Have you considered something similar, perhaps using get_iplayer's `--pvr-queue', etc., to add programmes of interest, and then telling it to `run' some or all of the queue items? > but have about three times as much data written to hd at some point. Yes, with my preferred choices, get_iplayer's writes to disk would be downloaded video downloaded audio video and audio combined into MP4 tagged MP4 so 3(v+a), and I'm not converting to 25 fps. > I could use spinning rust I guess. but that seems messy. Do you have it already available in the machine to use? That seems an easy compromise. You could also look at your SSD's specifications and whether you're striving too hard to avoid what might not be a problem with modern ones and their built-in wear levelling. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Richard, > > ffmpeg can produce PNGs, one per frame, and convert only a few > > specific seconds to avoid tens of thousands of them > > Would you first need to convert the H.264 or H.265 to raw video? If > one PNG is of an I-frame and the next is a P-frame or B-frame they are > bound to be different. No, AIUI ffmpeg(1) produces a PNG that's the whole assembled picture, just as if one hit `Pause' when watching TV. Any indication of how that was built up from the input video encoding is lost. ffmpeg -i foo.mp4 -t 1 foo-%04d.png -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Richard, > I think it was Nick Payne who said he had experimented with > re-encoding in HEVC (H.265) and found that the file size was the same > for 25fps as it was for 50fps, which led him to conclude that frames > were being duplicated to achieve 50fps. ffmpeg can produce PNGs, one per frame, and convert only a few specific seconds to avoid tens of thousands of them, and then adjacent frames could be compared in pairs to see if they are indeed identical; compare `ab', `bc', `cd'... There's a couple of variants for comparing images. compare foo.png bar.png x: gm compare -highlight-style assign foo.png bar.png -file x: -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Nick, > I had some of the coverage of the UK snooker championships ... > 4h58m match: D/L size 5.03Gb @ 25fps, output from Handbrake was 1.55Gb > 4h44m match: D/L size 10.2Gb @ 50fps, output from Handbrake was 1.48Gb Perhaps many frames(!) of snooker coverage isn't ideal for this comparison as it often is a static picture of the table. :-) -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Owen, > What do you mean this isn't a lossy transcoding? Is that aimed at me? Perhaps if you didn't top post, and instead wrote that under a quote of mine I'd know to which bit of the two ffmpeg invocations you were referring! :-) > How can ffmpeg go from 50fps to 25fps without losing anything? I don't think it can, and didn't suggest it could. It is lossy. I said the first, default, one wasn't, and that therefore it wasn't worth combining this extra, 50->25, one with it, as you would want to if both were lossy. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Jim, > > You could use get_iplayer's --command option to run a command to > > move each final file off tmpfs as the download is finished. Its > > --output affects all the intermediate files too, AIUI. > > The challenge for me is to work out how to get the fetched file to go > onto the tmpfs Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could `--output /tmp' and you should see its intermediate files, and the final file, only appear there. `--command' could then move that final file to the SSD, or run a conversion command that writes to the SSD and then removes the tmpfs input. As for altering the ffmpeg command that get_iplayer is using, I'm not sure that's worthwhile? It isn't doing any transcoding, just changing the container format, or splicing in better audio, that kind of thing. So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second lossy one that you'd prefer to combine with the first. I could be wrong, not knowing how to have ffmpeg do this conversion. When you find out, let the list know. :-) -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Jim, > How do I find out the command gip sends to ffmpeg to do its default > conversion as things stand? Try `--verbose'; that should print external commands that are run. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: no more hslv format ?
Hi Jim, > I've been discussing the 'loss' of the 1280x720 25fps version with > someone at the BBC. I miss those 1 GiB ~= 1 hour ones too. They were `just right'. > It has also set me wondering about arranging for gip to fetch to ram > storage and then convert that into a file on my main disc. Is this Linux? get_iplayer here uses the current working directory for all its large intermediate files. If that was a `tmpfs' filesystem then they would all sit in RAM, unless you've swap space and pressure caused pages from the filesystem to be swapped out. You could use get_iplayer's --command option to run a command to move each final file off tmpfs as the download is finished. Its --output affects all the intermediate files too, AIUI. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Cannot play downloads from get_iplayer!
Hi Paul, Peter S. Kirk wrote: > 2. running .mkv through ffmpeg to change to .mp4 Which doesn't transcode, just quickly pulls the streams out of the MKV and puts them in an MP4. ffmpeg -i in.mkv -c copy out.mp4 -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Steve Backshall - Nature's Microworlds - 2 Serengeti.mp4, b01l4906
Hi CJB, > is the overly-LOUD dramatic music. This is so loud that the narrator > cannot be heard The production companies paid by the BBC put `plinkity-plink' music over all the speech audio, not just narration, and not just to add drama. It seems to be for no good reason; similar to a presenter having to show they can walk and talk at the same time instead of just being a talking head. It all `adds interest'. Presumably because of lack of confidence in the spoken matter. Given, outside of iPlayer, I can watch a foreign film and choose the audio stream, e.g. German or dubbed English, and then choose the subtitle stream similarly, it would be nice if iPlayer offered two audio tracks with one having no needless muzak. This would be a bonus feature over broadcast, and the Beeb could gather stats on preference. They've the clout to insist production companies hand over the tinkle-free audio. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Steve Backshall - Nature's Microworlds - 2 Serengeti.mp4, b01l4906
Hi Tony, > Read this, and see what I mean > https://www.theregister.co.uk/2013/06/25/the_future_of_moving_images_the_eyes_have_it/ Thanks, interesting, though I didn't grasp it all on first reading. Don't suppose you know of a good article explaining why the narrator in BBC programmes is perceived as always being louder than the other voices despite Aunty insisting they're the same? :-) -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Unable to refresh
> So can we expect to see a new version of GiP if the Beeb has rejigged > its schedule pages? Yes. ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Unable to refresh
Hi Richard,
> > If they don't match anything then they normally remain and are
> > passed to get_iplayer anyway.
...
> > The argument with the glob is expanded into that and get_iplayer has
> > one argument, «R.steinway», that's used as a regexp. It's unlikely
> > to match any titles, e.g. «Resteinway».
...
> I clearly still need to think it through further.
Or, study my worked example above until it's understood. :-)
> I would have expected
> get_iplayer * --since 110
> to match 67 programmes, but it matches 0.
«*» is probably being expanded by the shell based on entries in the
current directory.
> I would have expected
> get_iplayer '*' --since 110
> to match 0 programmes but it matches 67.
get_iplayer, for some unknown reason, has
my @search_args = map { $_ eq "*" ? ".*" : $_ } @ARGV;
to break its consistency over argument handling. It treats these two
the same. A bad idea IMO.
get_iplayer '*'
get_iplayer '.*'
> If I put in an invalid regex
> get_iplayer *. --since 110
> get_iplayer '*.' --since 110
> both seem to reach Perl as a regex even though the first has a bash
> wildcard.
Again, the worked example predicts this and covers it. Create «a-dot-.»
in the current directory and try again.
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Unable to refresh
Hi Richard, > > > > If you are typing `get_player .* --since 70' into a Linux shell > > I have come across this in relation to bash "The characters *, ? and [ > are called glob characters or wild card characters. If an unquoted > argument contains one or more glob characters, the shell processes the > argument for file name generation. The glob characters are part of > glob patterns which represent file and directory names. These patterns > are similar to regular expressions, but differ in syntax, since they > are intended to match file names and words (not arbitrary strings). > The special constructions that may appear in glob patterns are: ... " > > What that seems to mean is that without quotes around *. a file name > or word can be matched by bash and with quotes an arbitrary string can > be matched as a regex. It is not clear to me why that matters. The shell is expanding globs before invoking get_iplayer, thus they're not seen by get_iplayer if they match anything. If they don't match anything then they normally remain and are passed to get_iplayer anyway. For the arguments get_iplayer does see, it decides to interpret some of them as regexps. «get_iplayer Railway» has no glob metacharacters to expand so one argument is passed to get_iplayer, it uses it as a regexp, it has no regexp metacharacters so effectively is a substring search of the titles. «get_iplayer R.*way» has a glob metacharacter, the «*», the shell looks at the current directory for entries starting «R.» and ending «way». There are none. The glob remains, unexpanded. get_iplayer has one argument, «R.*way» that it uses as a regexp. There's two regexp metacharacters, «.*», meaning zero or more of any character, used in the search. «get_iplayer R.*way» is run again, and again has a glob, the «*». This time, the current directory has «R.steinway» in it. The argument with the glob is expanded into that and get_iplayer has one argument, «R.steinway», that's used as a regexp. It's unlikely to match any titles, e.g. «Resteinway». To avoid glob expansion, quote the glob metacharacters, «get_iplayer 'R.*way'», and get_iplayer sees the regexp «R.*way». > One thing that does not appear to have happened is infinite recursion, > or even matching of additional programmes. Your unquoted «.*» on Linux would often expand to «. ..», and perhaps more if you've other `dot' files present. These are two regexps interpreted by get_iplayer. It prints titles matching either. Since anything matching the second is also matched by the first, you are seeing any title at least one character long. That's almost like «.*» and «^» except that a zero-length title won't be matched. > As for using a single quote ' as the strongest quote, I suspect the > documentation has used double quotes " for compatibility with Windows. Yes, I expect you're right. Fortunately, I've only had a little exposure to that in the days of DOS. :-) If Windows doesn't treat «^» specially then that could be used instead with no quoting needed. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Unable to refresh
Hi Richard, > > > If you are typing `get_player .* --since 70' into a Linux shell > > > then it will glob the `.*' and replace it with the expansion, e.g. > > > `. ..', unless it's quoted. > > The 3.09 release notes say, "get_iplayer no longer lists all > programmes when invoked without a search argument. If you wish to list > all programmes, you must now explicitly specify a wildcard search: > get_iplayer ".*" - note the quotes. > > "note the quotes" is in bold, but in this case it seems to give the > same results without. The instructions here say to add two and two, in bold, but I find multiplying them works just as well, and raising one to the power of the other. :-) «.*» glob'd by the shell to «.» and «..», and perhaps other things, then gives get_iplayer two or more regexps and it tries to match any of them. It's not your intent. Convention is to use the strongest quotes possible to ease the interpretation by the readers. For regexps, that's single quotes, so «get_iplayer '.*'». But «get_iplayer ^» will give the same results and needs no quotes. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Unable to refresh
Hi Richard,
> Am I doing something stupid, or is there a problem refreshing the cache?
...
> get_player .* --since 70
> shows 67 programmes
If you are typing `get_player .* --since 70' into a Linux shell then it
will glob the `.*' and replace it with the expansion, e.g. `. ..',
unless it's quoted. I don't see how that will be affecting your lack of
additions in the last 60 hours, but it clouds the problem. If you are
quoting it, then paste what you run to stop wasting our time. :-)
This will show when PIDs were added to your cache, in order.
sort -t\| -k16,16n -k1,1 ~/.get_iplayer/tv.cache |
gawk -F\| '
!/^#/ {
print strftime("%Y-%m-%d %T %z %a", $16, 1) " " $4 " " $3
}
'
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: The Coroner series 2 Episode 8 not available
Hi Roger, > tellyadict wrote: > > Nice idea but I'm not sure they would have thought too much about > > it. The strange thing here is they obviously had space to broadcast > > it because they had to fill the final slot on the friday with an > > episode from series 1 because they had 10 slots and after they > > scrapped episode 8 they had 9 episodes to fill them. > > Didn't they broadcast S1 Ep 1 instead of S2 Ep1, then followed by S2 > Ep 2-7/9-10, so there were 10 programmes for 10 slots? Exactly. http://lists.infradead.org/pipermail/get_iplayer/2018-March/011592.html has the gore. They published the schedule for S1 E1-10 up to a week in advance, then *after* E1 had broadcast they scheduled S2 E1-7,9-10 for the same remaining time slots as S1 E2-E10. Perhaps the BBC either consider it junk that needs no continuity, or the afternoon audience incapable of noticing. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: The Coroner series 2 Episode 8 not available
Hi Dave,
> Has anyone any idea why Episode 8 of the latest series of The Coroner
> is not available? it isn't available on bbc iplayer either?
The published data for that programme went very wrong. This is what my
tv.cache saw over time recently.
$ grep -h The.Coroner ~/.get_iplayer/tv.cache* |
> gawk -F\| '{print sprintf("s%de%02d", $8, $9), $5, $4, strftime("%Y-%m-%d
%T", $16), $3, $7}' |
> sort -u | sed 's/T/ /; s/:00+00:00//'
s1e01 2018-02-12 15:00 b06q54pd 2018-02-05 11:34:54 The Coroner: Series 1
First Love
s1e01 2018-02-12 15:00 b06q54pd 2018-03-08 09:14:57 The Coroner: Series 1
First Love
s1e02 2018-02-13 15:00 b06q553j 2018-02-05 11:34:54 The Coroner: Series 1
How to Catch a Lobster
s1e03 2018-02-14 15:00 b06q588j 2018-02-05 11:34:54 The Coroner: Series 1
That's the Way To Do It
s1e04 2018-02-15 15:00 b06q5c8m 2018-02-06 11:24:56 The Coroner: Series 1
The Fisherman's Tale
s1e05 2018-02-16 15:00 b06q5rq3 2018-02-06 11:24:56 The Coroner: Series 1
Gilt
s1e06 2018-02-19 15:00 b06qjdck 2018-02-12 07:30:53 The Coroner: Series 1
Capsized
s1e07 2018-02-20 15:00 b06qjfq0 2018-02-12 07:30:53 The Coroner: Series 1
The Salcombe Selkie
s1e08 2018-02-21 15:00 b06qjjlh 2018-02-12 07:30:53 The Coroner: Series 1
Napoleon's Violin
s1e09 2018-02-22 15:00 b06qjkc0 2018-02-12 07:30:53 The Coroner: Series 1
The Deep Freeze
s1e10 2018-02-23 15:00 b06qjl6j 2018-02-12 07:30:53 The Coroner: Series 1
Dirty Dancing
s2e01 2018-02-13 15:00 b0840227 2018-02-13 12:34:30 The Coroner: Series 2
The Drop Zone
s2e02 2018-02-14 15:00 b084022y 2018-02-14 08:47:52 The Coroner: Series 2
Perfectly Formed
s2e03 2018-02-15 15:00 b084024r 2018-02-14 08:47:52 The Coroner: Series 2
Those in Peril
s2e04 2018-02-16 15:00 b084025z 2018-02-14 08:47:52 The Coroner: Series 2
The Beast of Lighthaven
s2e05 2018-02-19 15:00 b0840277 2018-02-14 08:47:52 The Coroner: Series 2
The Captain's Pipe
s2e06 2018-02-20 15:00 b084dxth 2018-02-14 08:47:52 The Coroner: Series 2
Life
s2e07 2018-02-21 15:00 b084dzjy 2018-02-14 08:47:52 The Coroner: Series 2
Perfect Pair
s2e09 2018-02-22 15:00 b084f1zy 2018-02-14 08:47:52 The Coroner: Series 2
Pieces of Eight
s2e10 2018-02-23 15:00 b084f3nk 2018-02-14 08:47:52 The Coroner: Series 2
Crash
$
The first date-time is when it was to be broadcast; always 3pm. The
second is when that entry was added to tv.cache. s1e01 did actually
broadcast, but then all of the rest of s1 didn't. Not good if you only
invested time watching it on the basis of watching the series that was
already scheduled. (iPlayer support reference CAS-4806288-B3HT3L.)
s2e08 is missing from that list; they never scheduled it in this run of
repeats.
Ordering that list by scheduled broadcast time, and eliding duplicate
dates, gives
s1e01 2018-02-12 15:00 b06q54pd 2018-02-05 11:34:54 The Coroner: Series 1
First Love
s1e01 "" b06q54pd 2018-03-08 09:14:57 The Coroner: Series 1
First Love
s1e02 2018-02-13 15:00 b06q553j 2018-02-05 11:34:54 The Coroner: Series 1
How to Catch a Lobster
s2e01 "" b0840227 2018-02-13 12:34:30 The Coroner: Series 2
The Drop Zone
s1e03 2018-02-14 15:00 b06q588j 2018-02-05 11:34:54 The Coroner: Series 1
That's the Way To Do It
s2e02 "" b084022y 2018-02-14 08:47:52 The Coroner: Series 2
Perfectly Formed
s1e04 2018-02-15 15:00 b06q5c8m 2018-02-06 11:24:56 The Coroner: Series 1
The Fisherman's Tale
s2e03 "" b084024r 2018-02-14 08:47:52 The Coroner: Series 2
Those in Peril
s1e05 2018-02-16 15:00 b06q5rq3 2018-02-06 11:24:56 The Coroner: Series 1
Gilt
s2e04 "" b084025z 2018-02-14 08:47:52 The Coroner: Series 2
The Beast of Lighthaven
s1e06 2018-02-19 15:00 b06qjdck 2018-02-12 07:30:53 The Coroner: Series 1
Capsized
s2e05 "" b0840277 2018-02-14 08:47:52 The Coroner: Series 2
The Captain's Pipe
s1e07 2018-02-20 15:00 b06qjfq0 2018-02-12 07:30:53 The Coroner: Series 1
The Salcombe Selkie
s2e06 "" b084dxth 2018-02-14 08:47:52 The Coroner: Series 2
Life
s1e08 2018-02-21 15:00 b06qjjlh 2018-02-12 07:30:53 The Coroner: Series 1
Napoleon's Violin
s2e07 "" b084dzjy 2018-02-14 08:47:52 The Coroner: Series 2
Perfect Pair
s1e09 2018-02-22 15:00 b06qjkc0 2018-02-12 07:30:53 The Coroner: Series 1
The Deep Freeze
s2e09 "" b084f1zy 2018-02-14 08:47:52 The Coroner: Series 2
Pieces of Eight
s1e10 2018-02-23 15:00 b06qjl6j 2018-02-12 07:30:53 The Coroner: Series 1
Dirty Dancing
s2e10 "" b084f3nk 2018-02-14 08:47:52 The Coroner: Series 2
Crash
These were all `BBC One'. So it looks like they scheduled and broadcast
s1e01 by mistake, and then switched to s2e01, replacing s1e02, and so
one for the remaining nine slots. But that meant something had to go
and that was s2e08.
One more ordering, by the
Re: more creatures gret and small question
Hi Alan,
> I don't have access to the original schedule
I've quite a few tv.cache, backing it up every time it gets written.
They've all got a gap in them that morning, which is quite odd, before
11:45. I've marked those that start at the same time as another entry,
showing schedules change, with their group number.
$ cat ~/.get_iplayer/tv.cache* |
> awk -F\| '$5 ~ /^2018-03-02T[01]/ && $13 == "BBC Two" {print $5, $4, $3}'
|
> sort
2018-03-02T00:05:00+00:00 b09tmqlv Snooker: Welsh Open Highlights: 2018
2018-03-02T00:20:00+00:00 b017yznd Coast: Series 5 Reversions
2018-03-02T02:05:00+00:00 b09gsz3b Royal Recipes: Series 2
2018-03-02T11:45:00+00:00 b09trgmb Wanted Down Under Revisited: Series 11
2018-03-02T12:15:00+00:00 b09tn3l1 Caught Red Handed: Series 6
2018-03-02T13:00:00+00:00 b09tr342 Athletics: 2018
2018-03-02T14:15:00+00:00 b039sp5t Plan It, Build It
2018-03-02T14:45:00+00:00 b01406xp Coast: Series 5 Reversions
2018-03-02T14:45:00+00:00 b09w4403 Coast
2018-03-02T15:00:00+00:00 b07v7b6l Yes Chef: Series 1
2018-03-02T15:50:00+00:00 b086y32c A Place to Call Home: Series 4
1→ 2018-03-02T16:20:00+00:00 b0400tyf More Creatures Great and Small
1 2018-03-02T16:20:00+00:00 b07v6hdt Grand Tours of the Scottish Islands:
Shorts
2 2018-03-02T16:30:00+00:00 b09tmqw5 Snooker: Welsh Open: 2018
2 2018-03-02T16:30:00+00:00 b09tr21c Get Away for Winter: Series 1
2 2018-03-02T16:30:00+00:00 b09w8sgs Snooker: Welsh Open: 2018
2018-03-02T17:20:00+00:00 p03gk861 Greece with Simon Reeve
3 2018-03-02T17:50:00+00:00 b04871bz Coast: Series 5 Reversions
3 2018-03-02T17:50:00+00:00 b088rxvh Great British Railway Journeys: Series 8
2018-03-02T18:00:00+00:00 b0079285 Coast: Shorts
2018-03-02T19:00:00+00:00 b09tr344 Athletics: 2018
2018-03-02T19:30:00+00:00 b09tmrhv Snooker: Welsh Open: 2018
2018-03-02T19:57:00+00:00 b09ts2db MasterChef: Series 14
$
I wonder if that means obtaining the TV listings for the PVR is quite
sketchy, leaving gaps?
> but I would hazard a guess that when the Maybot announced she would be
> making her speech, the Beeb decided to reschedule Daily Politics,
> extend the Athletics and as a result More Creatures got bumped.
You're probably right. If they don't simply slide all the episodes
along one, but actually drop one episode from broadcast leaving a hole,
you'd think they would *still* make it available on iPlayer for those
that want continuity.
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: FAO BBC: Same Series and Episode, Differents PIDs and Description.
Hi David, > > b09wc2m1: Antique photography expert Brenton West repairs a > > camera that survived World War I. > > > > b09wbylq: A Boulle-work clock, a much-loved wheeled elephant and > > a 300-year-old desk are in the shop > > My assumption would be that they are different versions of the same > episode, with one cut for length for eg a repeat. You assume too much! :-) They're both 30 minutes, and completely different content. b09wc2m1 duration: 1800 desclong: Today in the Repair Shop, Jay Blades and the team bring three treasured family heirlooms, and the memories they hold, back to life. Resident antique photography expert Brenton West brings his expertise to bear on a camera that survived the First World War but hasn't taken a picture in over a hundred years, while leather expert Suzie adds a surprise touch that leaves the camera's owner Phil lost for words. Furniture restorer Will Kirk and horologist Steve Fletcher joins forces to get a 20th-century designer wooden screen back on its feet. And violin restorer John Dilworth works on a treasured instrument that spent World War II in Auschwitz concentration camp. b09wbylq duration: 1800 desclong: Today in the Repair Shop, Jay Blades and the team bring three treasured family heirlooms, and the memories they hold, back to life. Resident clockmaker Steve tackles an intricate Boulle-work clock that hasn't ticked for over 15 years. Soft toy restorers Julie and Amanda take on a jumbo-sized project in the shape of a threadbare, but much loved, wheeled elephant. And woodwork specialist Will get to grips with one of the Repair Shop's oldest ever assignments - a 300-year-old Georgian desk that is showing its age. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
FAO BBC: Same Series and Episode, Differents PIDs and Description.
Hi, There's been much weirdness recently in get_iplayer's tv.cache due to what's being published by the BBC AFAICS. The latest example, I have some older more complex ones to write up, is PIDs b09wc2m1 and b09wbylq are both S02E01 of _The Repair Shop_, but with different descriptions. The `short' one is b09wc2m1: Antique photography expert Brenton West repairs a camera that survived World War I. b09wbylq: A Boulle-work clock, a much-loved wheeled elephant and a 300-year-old desk are in the shop This confusion is also present on the BBC's site. http://www.bbc.co.uk/programmes/b09wc2m1 http://www.bbc.co.uk/programmes/b09wbylq -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Format of options file
Hi David, > binmode does still work AFAIK, but a more modern and flexible method > is to use the crlf I/O layer, which is documented here: > https://perldoc.perl.org/PerlIO.html > > Note however that an awful lot of perl code just doesn't bother. Windows stacks the `:crlf' layer by default. I *think* Richard's trying to avoid that because he wants his get_iplayer to use POSIX text files on Linux, as normal, and Windows. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Format of options file
Hi MacFH, > nor am I questioning the use of device drivers, obviously they make a > lot of programming sense. What I'm questioning is the wisdom, perhaps > that should be rank stupidity, You're saying Unix's core designers are stupid for choosing to stick with text files as lines terminated by LF. There's some Sherlock Holmes quote about `It takes talent to recognise genius...'. :-) > of using LF as the line-terminator, while treating a preceding CR as > part of the line text. That completely f*d backward compatibility > with what had gone before. Much of Unix broke orthodoxy, and Unix wasn't the first to punt device dependencies out of the logical file format and into a device driver that needed to know about the exceptions and oddities of the physical world. Nor was it the first major OS to use LF-terminated lines. > You say that it is C based, but my recollection of C's internal > representation is that strings are NULL terminated, They are, but that it not relevant. > and that also might have been a more logical choice of line-terminator > for Unix. Only if a C string was to be prohibited from holding more than one line of text. C allows the bytes "line 1\nline 2\n\0" as a NUL-terminated string. And only if a C string "foobar" could no long be split into two, "foo", "bar", without losing the fact that they are not both lines that have been read. C's standard library widely expects LF-terminated lines, and translates, as I've said twice now in replies to Richard, from the host's representation to that on reading and back on writing where it differs, e.g. CP/M and its descendants. You're drawing conclusions from sketchy knowledge and thus not able to follow through their implications. Sorry, but there's no point to this conversation. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Format of options file
Hi David, > Macs use \n like normal Unix machines. They used to use \r in the bad > old days before they went Unixy. Yes, you're right. Earlier in the thread I said I thought they used CR, but that was much earlier up to Mac OS 9. Mac OS X came along around 2000, was based on Unix, unlike 9, and switched to its LF line terminator. macOS is still version 10, the X Roman numeral, but has changed name to OS X, and now macOS along the way. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Format of options file
Hi Richard, > I suggested ignoring CR because they are there. Ideally they would > not be there. The files are internal to get_iplayer so they can be in > any format. Only if you don't want them to be native text files, editable on that system with a text editor by any user. And I thought you do because you have been editing them. > I would have thought it sensible to use LF as a line terminator for > get_iplayer's internal files across all operating systems. The > question is how the CR are getting into the files in Windows. I pre-empted this in the email you're replying to. Because C has lines terminated by LF, like Unix, C libraries on Windows convert CRLF to LF on reading a text file, and do the reverse on writing. Perl implements that. > If Perl can then handle the variants portably with chomp VARIABLE > that's fine, but it doesn't seem to be working properly at the moment. That's not what chomp() does. It removes a single trailing LF if it's present. The conversion of CRLF to LF on reading a text file is PerlIO(3perl)'s `:crlf' layer, stacked on Windows systems. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Format of options file
Hi MacFH, > 'Carriage Return', CR, meant precisely that - in other words, move > the slidable teleprinter carriage holding the paper to the right so > that the fixed print head is at the left margin position. LineFeed, > LF, also meant exactly that, rotate the carriage roller to move the > printing position to the next line. Those of us that used manual typewriters, that pre-date fangled teleprinters, will recognise the two actions that could be done by the lever to the left. It would feed the paper to the next line when pressed, and when continued to be pushed it would slide the carriage along so it returned to the start. It was the `Line feed/Carriage return lever'. > Arguably the oddballs are Linux and perhaps Macs, using only one. > This would mean that if their text was sent directly to a teleprinter, > the output would not be what is required, but of course these days > that would seldom if ever be done. But in those days of Multics and Unix that's precisely what was done with Unix text files, terminated by just LF, and yet the text didn't stagger across the page because a device driver was correctly the thing that knew what a particular device needed to be sent as bytes; the file format wasn't the place for that. Other characteristics include a timing delay to allow for the carriage to return before sending more bytes. > Linux could be argued to have subverted both codes, because not only > does a LF move the cursor to the next line of text, but an implicit CR > is also performed This happens in a terminal window for the same reason it happens on a teleprinter; the TTY is so configured and the device driver obeys. I can say this particular device doesn't need a CR sent to it before every LF. $ stty onlcr; printf 'foo\nbar\n'; stty -onlcr; printf 'foo\nbar\n' foo bar foo bar $ -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Format of options file
Hi Richard, > > Let's forget Mac for the moment. Linux text files are POSIX text > > files; zero or more lines, each terminated by a LF. See ascii(7). > > DOS ones use CR followed by LF at the end of each line. > > > > Thus a DOS text file looks like a text file to Linux, but one where > > the last character at the end of each line, just before the LF, is a > > CR, which is just another possible byte that could be within the > > line. If the operation you're doing doesn't mind that the CR is > > present then it works; reading the PID from the start of the line > > and using it as a key. But if you're using the data that runs to > > the end of the line then you'll pick up the unwanted CR and that may > > do things like corrupt the output. > > Essentially you seem to be saying I can't have a CR in a Linux text > file because it is non-standard. Nope, I'm not saying that. `Linux text files are POSIX text files; zero or more lines, each terminated by a LF.' Here's a one-line POSIX text file. $ sed -n l oneline one\rtwo\rthree$ $ cat oneline three $ CR is just another character, a perfectly valid one, that's data as opposed to the LF that's the line's terminator and meta-data. > It seems a strange standard that prevents data interchange rather than > facilitates it. It is an excellent standard and ensures data interchange between the hundreds of Unix text-processing tools that made Unix very popular. > One article I read mentioned that errors can be caused by strict > adherence to a standard so that the last line of data is ignored if it > does not end with a newline. Emacs is a non-Unix text editor that was ported to Unix, but kept its foreign text-file format that uses line separators, not terminators. Emacs users pollute the Unix filesystem with non-text files that have trailing bytes beyond the zero-or-more Unix text lines unless they change Emacs's defaults. Unix programs are not at fault here, nor is their `strict' adherence; why bother otherwise? > My approach would be to ignore the CR or to recognise three possible > end of record markers, but it seems that is not allowed. It certainly is allowed. It would be the Richard Text File Format. Programs could code to handle it. More precise specification would be required. Is the first line-ending encountered from the start of the file the one that must be used thence? Or may each line differ? RTTF is incompatible with reading the POSIX `oneline' text file above so conversion would be required, presumably by some means of escaping the CR? And escaping the escape character? And converting files that were already using the escape character to not mean escape. https://xkcd.com/927/ > I can exchange data in elaborate formats like DOCX, ODT or H.264 but I > can't as a text file. They are file formats designed for an application, or a standard for interchange. A text file is an operating-system file format, and they show their lineage. CPM and DOS let the needs of the hardware leak into the file format, wasting a byte each time. Before them, Unix, really Multics, bettered that by having a logical representation and leaving the hardware's needs to the device driver. I understand macOS, the latest name for it, has moved to POSIX's format. > What is the answer? How can Windows and Linux exchange data in text > files without requiring the user to run them through external > conversion programs? A program can code for one format everywhere, e.g. POSIX, but that then depends on editors that stick with POSIX format on non-POSIX systems. They exist and are becoming ever more common. Or it can do as Python as done and try to adapt on the fly, but that corrupts some POSIX text files on reading, which is why Python lets you disable it. Because C has lines terminated by LF, like Unix, C libraries on Windows convert CRLF to LF on reading a text file, and do the reverse on writing. Perl implements that. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Format of options file
Hi Colin,
> > > ') - using defaultalue for --ffmpeg-loglevel ('info
> >
> > That line above seems to be overprinting the «')» at the start of
> > the line suggesting there's an ASCII CR, carriage return, after the
> > `info' as if a Unix system was reading a POSIX text file of lines
> > ending in ASCII LF, linefeed, but being fed a DOS one with CR LF
> > pairs.
>
> Are we certain that the problem has been correctly analysed?
Yes, I did. :-)
> The line shown above appears to have a CR after info and before the
> quote sign, which would not be valid on either system.
Because the value is «info\r» and it's being printed surrounded by
«('...')».
WARNING: invalid value for --ffmpeg-loglevel ('$opt->{ffmpegloglevel}') -
using default\n
becomes
WARNING: invalid value for --ffmpeg-loglevel ('info
') - using default
becomes
') - using defaultalue for --ffmpeg-loglevel ('info
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Format of options file
Hi Richard, > What I said the download_history file was wrong. I have now looked at > it with a hex editor which allows me to view the whole file. Most of > it was written in Windows and has CRLF as a line terminator. The most > recent records, appended to the end, written in Linux, have LF as a > line terminator. The important thing is that the discrepancy does not > stop it working. The entries are indexed by pid. The script is able > to find the pid whether it is preceded by LF or CRLF, and identify the > programme as already in history. The mixed file works both in Linux > and in Windows. It probably doesn't. Let's forget Mac for the moment. Linux text files are POSIX text files; zero or more lines, each terminated by a LF. See ascii(7). DOS ones use CR followed by LF at the end of each line. Thus a DOS text file looks like a text file to Linux, but one where the last character at the end of each line, just before the LF, is a CR, which is just another possible byte that could be within the line. If the operation you're doing doesn't mind that the CR is present then it works; reading the PID from the start of the line and using it as a key. But if you're using the data that runs to the end of the line then you'll pick up the unwanted CR and that may do things like corrupt the output. $ seq 3 1 2 3 $ seq 3 | grep '^2$' 2 $ seq 3 | sed 's/$/\r/' 1 2 3 $ seq 3 | sed 's/$/\r/' | grep '^2$' $ $ seq 3 | sed 's/$/\r/' | sed -n l 1\r$ 2\r$ 3\r$ $ The first grep works. Appending CR with sed doesn't change the apparent appearance, but it's there as a byte so the second grep fails to find the line that contains just `2'. sed's `l' command shows up the problem. So the new lines added by Linux to the end of the history file might well look like one long, unterminated as there's no CR LF, line to Windows. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
FAO BBC: Double-encoded UTF-8 in Programme's JSON.
Hi, I noticed get_iplayer showing Rothaà Móra an tSaoil: Series 1 and wondered if it was a bug, but the BBC's JSON has $ curl -sS https://www.bbc.co.uk/programmes/b09w6dhm.json | > grep -o '"Roth[^"]*"' "Rotha\u00c3\u00ad M\u00c3\u00b3ra an tSaoil" "Rotha\u00c3\u00ad M\u00c3\u00b3ra an tSaoil" $ and get_iplayer is correctly showing U+c3 and U+ad after `Rotha'. The problem is the BBC have taken a UTF-8 encoding of the intended rune and encoded it again as UTF-8. $ iconv -f utf-8 -t ucs-2be <<<$'\xc3\xad \xc3\xb3' | > od --endian=big -tx2 000 00ed 0020 00f3 000a 010 $ Thus the title is meant to be $ printf 'Rotha\u00ed M\u00f3ra an tSaoil\n' Rothaí Móra an tSaoil $ Can a BBC lurker please see if they can stop it happening. Thanks. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: What do you guys make of this? iPlayer/VPN strangeness
Hi John, > Running GiP 3.09 on Linux Mint, connecting using a VPN for privacy > purposes. ... > Same VPN IP address was being used throughout. So you're connecting to a VPN so your traffic to the BBC appears to come through the VPN? What routes to the BBC has the VPN got? More than one? Changing with time? It seems more detail about the nature of the VPN is needed? -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: 'My indexing' broken by 3.07
Hi Charles, > > Yes. `^' also suffices. > > Interesting. I wonder if 'match beginning of the line' is less > expensive internally? Perl's regexp engine is historically extremely good at spotting optimisations, and some of those details can be seen with its -D option if perl is compiled suitably, e.g. /x.*foo$/ might decide the minimum length is four and it must end with `foo' before attempting to find an `x'. But in this case, I think `^' is cheaper. $ for p in ^ '.*'; do > for n in 100 1000 1; do > seq $n | > perf stat -e instructions \ > perl -ne "/$p/" > done > done |& > grep instructions:u 2,588,069 instructions:u 4,947,485 instructions:u 28,715,945 instructions:u 2,600,183 instructions:u 5,089,466 instructions:u 30,189,787 instructions:u $ Re-arranged, that's n /^//.*//.*/ 100 2,588,069 2,600,183 +12,114 1,000 4,947,485 5,089,466+141,981 10,000 28,715,945 30,189,787 +1,473,842 -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: 'My indexing' broken by 3.07
Hi Charles, > > ] wildcard search: get_iplayer ".*" - note the quotes. > > Thanks so much for that Mark. That looks like a regex. Is it, do you know? Yes. `^' also suffices. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Modes and best quality
Hi Nick, > > > I have noticed, on numerous pids, that if I download them using > > > --tvmode=best then the highest quality 25fps stream that is found > > > is dvfxhigh (704x396 25fps ). > > > > > > However, if I explicitly use --tvmode=hlshd, then I get the > > > 1280x720 25fps stream. I'm paying a bit more attention this time. The cmp_modes() function scores a mode and compares two modes by their score. Unfortunately, there's no `score_mode()' function so that the list of modes that's verbosely output can be augmented by their score. However, I added main::logger "NICK: ranks: $x $rank_x v. $y $rank_y\n"; just before the last line, the `<=>', of cmp_modes and then prefixed the `INFO' lines that were already output for PID `b09hlzbb'. score 20101 dvfhd1 gip_dvf_iplayer_5070 dash h264 1280x720 50fps 5070kbps stream mf_limelight_uk_dash/2 20102 dvfhd2 gip_dvf_iplayer_5070 dash h264 1280x720 50fps 5070kbps stream mf_akamai_uk_dash/1 20201 dvfsd1 gip_dvf_iplayer_2812 dash h264 960x540 50fps 2812kbps stream mf_limelight_uk_dash/2 20202 dvfsd2 gip_dvf_iplayer_2812 dash h264 960x540 50fps 2812kbps stream mf_akamai_uk_dash/1 20301 dvfxsd1 gip_dvf_iplayer_1604 dash h264 960x540 25fps 1604kbps stream mf_limelight_uk_dash/2 20302 dvfxsd2 gip_dvf_iplayer_1604 dash h264 960x540 25fps 1604kbps stream mf_akamai_uk_dash/1 20501 dvfhigh1gip_dvf_iplayer_1570 dash h264 704x396 50fps 1570kbps stream mf_limelight_uk_dash/2 20502 dvfhigh2gip_dvf_iplayer_1570 dash h264 704x396 50fps 1570kbps stream mf_akamai_uk_dash/1 20601 dvfxhigh1 gip_dvf_iplayer_827 dash h264 704x396 25fps 827kbps stream mf_limelight_uk_dash/2 20602 dvfxhigh2 gip_dvf_iplayer_827 dash h264 704x396 25fps 827kbps stream mf_akamai_uk_dash/1 21001 dvflow1 gip_dvf_iplayer_437 dash h264 512x288 25fps 437kbps stream mf_limelight_uk_dash/2 21002 dvflow2 gip_dvf_iplayer_437 dash h264 512x288 25fps 437kbps stream mf_akamai_uk_dash/1 50101 hlshd1 gip_hls_iplayer_2439 hls h264 1280x720 25fps 2439kbps stream akamai_hls_open/10 50401 hlsvhigh1 gip_hls_iplayer_1496 hls h264 832x468 25fps 1496kbps stream akamai_hls_open/10 60101 hvfhd1 gip_hvf_iplayer_5714 hls h264 1280x720 50fps 5714kbps stream mf_bidi_uk_hls/3 60102 hvfhd2 gip_hvf_iplayer_5714 hls h264 1280x720 50fps 5714kbps stream mf_limelight_uk_hls/2 60103 hvfhd3 gip_hvf_iplayer_5714 hls h264 1280x720 50fps 5714kbps stream mf_akamai_uk_hls/1 60201 hvfsd1 gip_hvf_iplayer_3320 hls h264 960x540 50fps 3320kbps stream mf_bidi_uk_hls/3 60202 hvfsd2 gip_hvf_iplayer_3320 hls h264 960x540 50fps 3320kbps stream mf_limelight_uk_hls/2 60203 hvfsd3 gip_hvf_iplayer_3320 hls h264 960x540 50fps 3320kbps stream mf_akamai_uk_hls/1 60301 hvfxsd1 gip_hvf_iplayer_2040 hls h264 960x540 25fps 2040kbps stream mf_bidi_uk_hls/3 60302 hvfxsd2 gip_hvf_iplayer_2040 hls h264 960x540 25fps 2040kbps stream mf_limelight_uk_hls/2 60303 hvfxsd3 gip_hvf_iplayer_2040 hls h264 960x540 25fps 2040kbps stream mf_akamai_uk_hls/1 60501 hvfhigh1gip_hvf_iplayer_2004 hls h264 704x396 50fps 2004kbps stream mf_bidi_uk_hls/3 60502 hvfhigh2gip_hvf_iplayer_2004 hls h264 704x396 50fps 2004kbps stream mf_limelight_uk_hls/2 60503 hvfhigh3gip_hvf_iplayer_2004 hls h264 704x396 50fps 2004kbps stream mf_akamai_uk_hls/1 60601 hvfxhigh1 gip_hvf_iplayer_1216 hls h264 704x396 25fps 1216kbps stream mf_bidi_uk_hls/3 60602 hvfxhigh2 gip_hvf_iplayer_1216 hls h264 704x396 25fps 1216kbps stream mf_limelight_uk_hls/2 60603 hvfxhigh3 gip_hvf_iplayer_1216 hls h264 704x396 25fps 1216kbps stream mf_akamai_uk_hls/1 60701 hvfstd1 gip_hvf_iplayer_1069 hls h264 640x360 25fps 1069kbps stream mf_bidi_uk_hls/3 60702 hvfstd2 gip_hvf_iplayer_1069 hls h264 640x360 25fps 1069kbps stream mf_limelight_uk_hls/2 60703 hvfstd3 gip_hvf_iplayer_1069 hls h264 640x360 25fps 1069kbps stream mf_akamai_uk_hls/1 61001 hvflow1 gip_hvf_iplayer_803 hls h264 512x288 25fps 803kbps stream mf_bidi_uk_hls/3 61002 hvflow2 gip_hvf_iplayer_803 hls h264 512x288 25fps 803kbps stream mf_limelight_uk_hls/2 61003 hvflow3 gip_hvf_iplayer_803 hls h264 512x288 25fps 803kbps stream mf_akamai_uk_hls/1 71101 subtitles1 captions http stream mf_limelight_uk_plain/20 71102 subtitles2 captions http stream mf_akamai_uk_plain/10 71103 subtitles3 captions http stream mf_limelight_uk_plain/20 From this list, and the base amounts that comprise a score in that function, you might be able to see why it thinks it's giving you the `best'; I expect x×y resolution isn't the central criteria. --
Re: Modes and best quality
Hi Nick, > I have noticed, on numerous pids, that if I download them using > --tvmode=best then the highest quality 25fps stream that is found is > dvfxhigh (704x396 25fps). > > However, if I explicitly use --tvmode=hlshd, then I get the 1280x720 > 25fps stream. I too am unclear on how the `better', `best', etc., map to modes. You might want to experiment with the --fps50 option as that seems to influence the definition of `best' and friends. > This behaviour does not accord with the documentation at > https://github.com/get-iplayer/get_iplayer/wiki/modes Yes, I thought that was out of date when skimming it a couple of weeks ago. I think get_iplayer changed partly because 25 fps is the `best' most folks want for throwaway TV fodder. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: parser error
Hi Graham,
> ERROR: Failed to load subtitles:
> :7: parser error : Char 0x0 out of allowed range
...
> It is still a small % but frequent enough to be annoying if you rely
> on subtitles to fully follow the speech.
I haven't tried this, and I'm looking at 3.06 rather than 3.07, but if
you find these lines in your get_iplayer script,
sub ttml_to_srt {
my $ttml = shift;
and add after them
$ttml =~ y/\0//d;
then that will delete any ASCII NUL bytes from the obtained URL before
attempting to parse it as XML. Hopefully.
But really, report each occurrence to the BBC because they're shipping
invalid XML and they need to find out why they keep doing it and fix the
cause.
(When looking at this, I also noticed a --subsraw option that saves the
URL's content in .../foo.ttxt before attempting to XML-parse it.)
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: lastbcastrel
Hi Charles, > Is it just me or is 'lastbcastrel' a thing of the past? Seems to have > disappeared from the metadata Yes, it's not in --info output here, nor mentioned in get_iplayer. `firstbcastrel' is the opposite on both counts. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: With 3.07 .dash.m4v left behind
Hi Richard, > As well as separating the two files in the unlink statement at 5988 you > could try inserting a delay between them as Ralph suggested. No, I didn't. :-) I did suggest adding a delay *before* attempting to unlink to allow all the bits of ffmpeg hanging around after the main thread has exited to themselves exit. If that's done, and the problem doesn't reoccur after enough attempts compared to its previous frequency, then that would suggest I'm right about the cause. Best of all would be to collect `$!' when one of the unlinks returns false as I suspect that will be a specific `file is open' error. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: With 3.07 .dash.m4v left behind
Hi Richard,
> { unlink( $audio_file, $video_file ); }
> acts on two files together.
That's fine. If removing either of those fails then unlink returns
false, setting `$!' to an error. You only need to do them separately if
you want to determine which had the error, and collect possibly
different reasons for the errors.
This problem and the other rename one look to be race conditions inside
ffmpeg; I'm suggesting it exits allowing get_iplayer to continue before
all of ffmpeg's threads have finished so some of the files it had open
are still open. As a race condition, it isn't likely to be trivially
repeatable, e.g. same PID.
Examination of ffmpeg might show this is possible. There may be ffmpeg
bugs already reported on it. And I suspect Process Monitor or something
similar could track the file and thread activity across all the
processes so post-inspection can confirm; but that's Windows and I
don't know it.
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: With 3.07 .dash.m4v left behind
Hi Nick, > > > A quick skim of all the `unlink' function calls show none have > > > their return value checked. ... > Doesn't look as though using --verbose gives any information to help > with the problem. Well, it wasn't going to include anything about the unlinks returning errors and what those errors were. :-) -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: No Modes for b05p6gj6 and Others.
Hi Richard, > Your complaint seems to be that a programme appeared in your cache > when it ought not to have done because it had not been broadcast > within the last 30 days. Yes. I think the BBC published information that s09 would be made available, and then stopped publishing that information. > I don't pay much attention to the cache, and I don't even bother to > make sure my cache is complete. I use nothing but the cache, and I make sure the cache is complete. :-) To do otherwise would waste much time having me click around the organ-grinder's GUI and yet still miss new programmes. As it is, I get a succinct list of what's newly available, excluding: what's in the download history; those already in the PVR; and all series and individual episodes I've declared `boring'. Series I've labelled `favourites' have their entries highlighted. It doesn't take long on a Monday, when the BBC publish most of their schedule, to run `./whatsnew' and update the PVR, the boring list, etc. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: With 3.07 .dash.m4v left behind
Hi Nick, > > > I assume this is meant to be deleted once the conversion to mp4 > > > has been completed, but this isn't happening consistently. > > > > Does Windows allow a file to be deleted if a process still has it > > open? Unix does. Perhaps this is another manifestation of my guess > > in > > http://lists.infradead.org/pipermail/get_iplayer/2017-November/011371.html > > In that previous case (temp file not able to be renamed) a permissions > error appears on the console. In this case no problem is indicated at > all. That just means get_iplayer is ignoring the failure to remove the file rather that report it to you. A quick skim of all the `unlink' function calls show none have their return value checked. You might want to adjust `unlink $foo;' to unlink $foo or main::logger "WARNING: unlink $foo failed: $!\n"; `$!' will hopefully give a Windows-specific error indicating why the removal failed, e.g. `file open'. > > by people that know Windows and can track what files are open when. Google suggests FileMon did this in the past, and it's been superseded by Process Monitor. https://docs.microsoft.com/en-us/sysinternals/downloads/procmon -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: No Modes for b05p6gj6 and Others.
Hi Richard, > What matters is that the episodes you want are not available in the > iPlayer. And as I said before, that might just be because of the same underlying problem that's affecting the data get_iplayer retrieves. :-) I've deleted s09 from tv.cache and refreshed it. They don't re-appear. s09e01's b05p6gj6 wasn't in the large Monday refresh at 2017-11-20 09:30:40 but appeared in Tuesday's refresh of 2017-11-21 12:14:08. Seemingly by mistake? -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: No Modes for b05p6gj6 and Others.
Hi Richard, > If you go to > https://www.bbc.co.uk/iplayer/episodes/b006t6m6?suggid=b006t6m6 it > lists 13 available episodes. Series 9 episode 1 is not one of them. Agreed, and I'd checked that, but perhaps the same problem with s09 at the Beeb is affecting iPlayer too. :-) > The expiry dates are calculated by get_iplayer, and are not the BBC's > expiry dates. Ah, perhaps that changed with the BBC dropping some of the metadata available. > Some programmes are only available for 7 days, but get_iplayer will > show a date 30 days ahead. But that can't account for this because s09e04 is still within the seven days. senum available pid modes A s09e01 2017-11-28T06 b05p6gj6 A s09e02 2017-11-29T06 b05q11ms A s09e03 2017-11-30T06 b05qshpy A s09e04 2017-12-01T06 b05rd726 -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
No Modes for b05p6gj6 and Others.
Hi,
Here's a summary of the output of get_iplayer's --info on these PIDs.
senum available pid modes
s08e01 2017-11-20T06 b03qgtzn original
s08e02 2017-11-21T06 b03sg2ft original
s08e03 2017-11-22T06 b03t7wh7 original
s08e04 2017-11-23T06 b03tzm15 original
s08e05 2017-11-24T06 b03vlcq7 original
s08e06 2017-11-27T06 b03w7y3c original
s07e01 2017-11-28T06 b01nzqyq original
A s09e01 2017-11-28T06 b05p6gj6
s07e02 2017-11-29T06 b01p2s95 editorial
A s09e02 2017-11-29T06 b05q11ms
s07e05 2017-11-30T06 b01pdxjk original
A s09e03 2017-11-30T06 b05qshpy
s07e06 2017-12-01T06 b01pjj8b original
A s09e04 2017-12-01T06 b05rd726
s10e01 2017-12-04T06 b06pmgdn original
s10e02 2017-12-05T06 b06q5gzr original
T s10e03 2017-12-06T06 b06qrk6m original
s10e04 2017-12-07T06 b06rcymj
s10e05 2017-12-08T06 b06s2w7l
s10e06 2017-12-11T06 b06sg6kb
`T' marks today, so it's reasonable that ones after don't have any modes
as their `available in the future.
`A' marks those I think are AWOL. They are s09e0{1..4} so I think the
BBC has broken series 9? Note, it's not because they were on a short
fuse and have expired. b05p6fcy's --info says `expires: in 21 days 18
hours (2017-12-28T06:00:00+00:00)'.
I'm hoping others on the list also find b05p6gj6 can't be retrieved, so
it's not just me, and that a lurker can put his BBC hat on and give the
idiot lantern a kick. :-)
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: With 3.07 .dash.m4v left behind
Hi Nick, > I assume this is meant to be deleted once the conversion to mp4 has > been completed, but this isn't happening consistently. > > OS is Windows 10 x64. Does Windows allow a file to be deleted if a process still has it open? Unix does. Perhaps this is another manifestation of my guess in http://lists.infradead.org/pipermail/get_iplayer/2017-November/011371.html that got no replies, e.g. by people that know Windows and can track what files are open when. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Cannot download b09hzwsn
Hi Nick, > Yes, that was the problem. But I could download it fine with iPlayer > on my phone at the same time as GiP was not seeing any modes... I *think* a `--get --pid $pid' searches for $pid through the existing, already downloaded, cache of what's available. It could be that cache was too old to include the last _Pot Black_, but not old enough to be automatically refreshed. The `--refresh' option forces that for this week and previous weeks IIRC. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Off-Topic forum error
Hi Richard, > PS The whole forum now seems to be affected. Works for me, as of now, e.g. https://forums.squarepenguin.co.uk/thread-1593-post-7103.html#pid7103 -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Linux converting from manual installation to PPA woes.
Hi John, > But which is correct? Use the latest and greatest? /usr/bin/ffmpeg -version /usr/local/bin/ffmpeg -version -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Linux converting from manual installation to PPA woes.
Hi John, > However, if I sudo, it works Don't do that. :-) This is Ubuntu? I wonder where the core dump is ending up. You could `ulimit -c' to see the current limit on core-file size for that shell and then `ulimit -c unlimited' if it's not that already. Run `./get_iplayer --verbose' to see if that gives more of a clue how far it's getting. If not, change that option to `--debug'. If nothing is clearer, run it under strace to help show what process is suffering. It should still suffer the SEGV. strace -fe execve get_iplayer -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Linux converting from manual installation to PPA woes.
Hi John, > so I deleted the executable in /usr/local/bin. However even though > 'which get_iplayer' returns /usr/bin (the location where the PPA > version is installed) when I try to run it, something is still > pointing to /usr/local/bin. How do you know this? Try `type get_iplayer' as an alternative to which(1). That's built-in to the bash shell you're using and tells you what it will do. Is your $PATH set up to have /usr/local/bin before /usr/bin? What happens if you give an explicit path to run it, i.e. /usr/bin/get_iplayer? -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Sometimes get file rename error at end of download
Hi, I missed the start of this thread, but jumping in midway... > > > INFO: Command: "ffmpeg" "-loglevel" "fatal" "-stats" "-y" "-i" ... > > > INFO: Command exit code 0 (raw code = 0) > > > ERROR: Could not rename file: > > > D:\Users\Nick\Videos\iplayer\Peaky_Blinders-s04e02-Heathens.partial.mp4 > > > ERROR: Destination file name: > > > D:\Users\Nick\Videos\iplayer\Peaky_Blinders-s04e02-Heathens.mp4 > > > INFO: Skipping all versions of this programme IIRC Windows doesn't allow a file to be rename if a process has it open. I predict that ffmpeg is multiple threads/processes and the one get_iplayer started, and sees exit successfully with `0', isn't the "last man standing". One or more of the other ffmpegs are still around, with a handle on that partial file, causing the rename to fail. I'd consider this an ffmpeg bug; its main process should only exit when it knows the others it has started have finished. Meanwhile, you may want to edit get_iplayer to add a `sleep 3.14;' between the ffmpeg exit and the rename to make the race condition far less likely. If that works, let the list know. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Mode sizes - but somewhat OT
Hi Charles, > I'm looking at the modesizes for I Know Who You Are and note the > lowest is c. 400MiB. I was wondering what sort of mode size we're > talking about when watching that as default on an Android phone's > iPlayer? I mean - approximately. No doubt the small device size should > lower it? get_iplayer's --info for PID b09ffrlp shows dvflow1 and dvflow2 approximated as 248MiB. I downloaded b09ffrlp using ./get_iplayer --get --tvmode dvflow 3915 and got 318,790,871 bytes for 01:19:18.04 of video and audio. That's 304.02 MiB. `ffmpeg -i *b09ffrlp*.mp4' says Stream #0:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709), 512x288 [SAR 1:1 DAR 16:9], 437 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default) of which 512x288 is probably of most interest to you. You should have a read of this page too, in particular https://github.com/get-iplayer/get_iplayer/wiki/modesref#tv-modes -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Searching for programmes which start with a number
Alan wrote: > get_iplayer --type radio "^1834$" For those that don't know regexps but want to have "wildcarded" searches consisting of one or more strings that must appear in order, e.g. `money', `mouth', `13', join them together with `.*', that means any character repeated zero or more times, to give `money.*mouth.*13'. If you only want those that start with the first string then prepend `^', and for those that end with the last string, append `$'. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problem with 3.06 PVR
Hi Richard,
> The way the cache is refreshed has changed so much recently that I may
> be confusing a historical regime with the present one.
Yes, the arms race does make it tricky to keep up. I only dig when
something stops working.
> My understanding is that the 30 day limit refers to how long entries
> remain in the cache. Programmes older than 30 days are discarded.
It currently controls how many whole-weeks back we go in retrieving
schedules like https://www.bbc.co.uk/schedules/p00fzl6n/2017/w$w for BBC
One, where $w is the week number of the year, e.g. today is in week 45.
Entries in the cache are purged by get_links() if they don't have an
"expires" time, or it's before "now", or its "timeadded" is more than 30
days ago. That 30 days is fixed and defined as $min_timeadded.
> The schedules only exist for this week and last week, so it is not
> possible to add programmes more than 2 weeks old.
I've just got all those for {01..45} and they're all available. I think
it's just get_iplayer doesn't want to slow things down by getting the
older ones, even though they obviously contain programmes that could
still be available, e.g. _This Week_ is typically online for a year.
(They can't give it away!) This suggests if one comes back from a
three-month cruise that overriding the 30-day limit could be useful.
> The v3.00 release notes
> https://github.com/get-iplayer/get_iplayer/wiki/release300to309#release300
> state,
> The slower updates are compensated for by the fact that the cache is
> now updated only once per calendar week (Mon-Sun). The cache will be
> updated the first time get_iplayer runs during the week.
>
> My cache still seems to refresh every 4 hours, so I may have got it
> completely wrong.
I think it's changed again since 3.00. The --expiry option defaults to
four hours now; if the cache is older than that then it is refreshed.
(It's also refreshed if explicitly requested, or it doesn't exist.) I
always specify a very high --expiry option because I explicit refresh
the cache once each morning, without waiting for it to finish, and then
don't want it delay any of my later runs of get_iplayer during the day.
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problem with 3.06 PVR
Hi Richard, > get_iplayer --cache-rebuild --type=tv,radio ... > You can only add the last two weeks anyway. --cache-rebuild implies --refresh-limit=30, unless that's already been set to some other value. 30 (days) is the current maximum. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problem with 3.06 PVR
C E Macfarlane wrote: > Note that perl uses the TMPDIR environment setting, and its absence > can lead to some problems with memory, particularly with embedded > devices, but perhaps also with Linux PCs. Presuming you have a /tmp > directory, you need to include in one of, in order of preference ... If TMPDIR isn't set then /tmp will be the default AIUI so the only reason to normally set it is when /tmp isn't big enough, e.g. it's a tmpfs that's using RAM, and then it needs to be set to somewhere else. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problem with 3.06 PVR
Hi Jon, > 501 Protocol scheme 'https' is not supported (LWP::Protocol::https not > installed) > Content-Type: text/plain > Client-Date: Fri, 10 Nov 2017 00:51:01 GMT > Client-Warning: Internal response > > LWP will support https URLs if the LWP::Protocol::https module > is installed.\n Well, that's very clear. :-) If you've time, perhaps you could urge get_iplayer's author, on the forums I think, to give better feedback to the user on those errors, perhaps conditionally, as LWP have obviously gone to the effort to make the information available and it will save others time in the future. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problem with 3.06 PVR
Hi Jon,
> I'm struggling to know where to look next - there isn't any real clue
> in the program output - is there a way to find out *why* it fails "to
> download programme schedule
> http://www.bbc.co.uk/6music/programmes/schedules/last_week; etc.?
The code ignores the error and just tries again up to a given number of
retries. Try modifying the code to give more information.
Make a back up copy of your get_iplayer script. Edit it and find
`sub request_url_retry' at the start of a line. A few paragraphs down,
add the `$res->dump();' line where it's shown below, then save the file.
for ($i = 0; $i < $retries; $i++) {
$res = $ua->request( HTTP::Request->new( GET => $url ) );
if ( ! $res->is_success ) {
$res->dump();
logger $failmsg if $i == $retries - 1;
} else {
logger $succeedmsg;
last;
}
}
You should now get output on those failures.
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: --history usage
Hi Richard, > I eventually found it was a Unix Epoch Timestamp, the number of > seconds since 1 January 1970. It can be converted to a date by > formatting a cell containing =(((E1/60)/60)/24)+DATE(1970,1,1) as a > date. Yes, that works most of the time and is good enough. I think the seconds-since-epoch time in the file is GMT, i.e. +. And it doesn't have any leap seconds. :-) -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: --history usage
Hi Richard,
> What is the significance of the index numbers returned by a search
> with --history?
They are line numbers, one based, in ~/.get_iplayer/download_history.
> Is there any easy way of getting the complete records?
It's a simple text file of one record per line, with `|'-separated
fields.
$ awk -F\| '{print $6}' ~/.get_iplayer/download_history |
> sort | uniq -c | sort -n
1 dafhigh1
1 flashaaclow1
1 flashvhigh2
1 hlsvhigh1
1 hvflow1
4 flashhigh1
10 flashhd2
28 flashaacstd1
31 hvfxsd1
148 flashvhigh1
312 hlshd1
1278 flashhd1
$
--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy
___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Download speed and progress indicator
Hi Richard, > Does Perl distinguish between desktop and laptop machines? No. :-) > Any ideas on what is causing the different behaviour? Do you physically sit at all of these machines' screens, or access some over a network? What OS are they running? If Linux, are you using the same terminal emulator on all of them? xterm(1) is a good standby for testing. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problem with 3.06 PVR
Hi again John, > > > > INFO: Indexing tv programmes (concurrent) > > > > .Mojo::Reactor::Poll: I/O watcher failed: SSL_ca_file > > > > SCALAR(0xfeb188) does not exist at /usr/share/perl5/IO/Socket/SSL.pm > > > > line 1642. > > have you tried `get_iplayer --refresh --no-index-concurrent'? I forgot to mention, Homebrew users on Mac OS suffered these until the recipe was updated to depend on later versions of Perl modules. https://github.com/Homebrew/homebrew-core/issues/19956 By using --no-index-concurrent, I think Mojo stops being used and thus the Perl-module versions won't matter, hopefully. I use --no-index-concurrent anyway because I don't want to install the extra Mojo dependencies. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
Re: Problem with 3.06 PVR
Hi John, > I've discovered that if I update the cache from a terminal window > using 'sudo get_iplayer --refresh' instead of just 'get_player > --refresh' everything works fine. > So it's a permissions issue, for some reason. get_player 3.06 running > on Linux Mint 17.3 "Rosa". It might not be a permissions problem, and you should not have to run as root with sudo. Not unless you've messed things up in the past by dabbling somehow with root. Based on your previous error report that you quoted, > >> INFO: Indexing tv programmes (concurrent) > >> .Mojo::Reactor::Poll: I/O watcher failed: SSL_ca_file > >> SCALAR(0xfeb188) does not exist at /usr/share/perl5/IO/Socket/SSL.pm > >> line 1642. have you tried `get_iplayer --refresh --no-index-concurrent'? -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy ___ get_iplayer mailing list get_iplayer@lists.infradead.org http://lists.infradead.org/mailman/listinfo/get_iplayer
