Re: no more hslv format ?

[Budge]

On 20/05/18 23:32, SquarePenguin wrote:

On 20/05/2018 23:23, Budge wrote:

Either ffmpeg or Handbrake will do, whichever works!


There is a post on the forum where someone explains how they use 
Handbrake to do it:


https://forums.squarepenguin.co.uk/thread-1786-post-7781.html#pid7781

There is also this, using ffmpeg, (but it's a little simplistic):

https://forums.squarepenguin.co.uk/thread-1765.html?highlight=50fps

Which leads here (which shows more options):

https://trac.ffmpeg.org/wiki/Encode/H.264


Hi and many thanks for the links.
I had seen and read all but the first before posting but not fully 
understood what I should be doing.  I am not lazy, just too busy to 
concentrate on this at present and was hoping to be able to copy a 
suitable command line example.  Will return to topic and post my results 
as soon as I can but if others are ahead of me please do share.

Thanks again,
budge

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[SquarePenguin]

On 20/05/2018 23:23, Budge wrote:

Either ffmpeg or Handbrake will do, whichever works!


There is a post on the forum where someone explains how they use 
Handbrake to do it:


https://forums.squarepenguin.co.uk/thread-1786-post-7781.html#pid7781

There is also this, using ffmpeg, (but it's a little simplistic):

https://forums.squarepenguin.co.uk/thread-1765.html?highlight=50fps

Which leads here (which shows more options):

https://trac.ffmpeg.org/wiki/Encode/H.264


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Budge]

On 20/05/18 11:21, Alan Milewczyk wrote:

On 20/05/2018 09:56, Mark Carroll wrote:

On 20 May 2018, Alan Milewczyk wrote:


On 19/05/2018 22:27, Mark Carroll wrote:

On 17 May 2018, Budge wrote:


I have tried to follow this thread as I have just found my download of
The Bridge file size and resolution are less than before.  Upgraded to
3.14.0 and forced a new download and file size is more than twice the
first.  I really would like to get back to 1 hour > approx 1GB and the
advice from iz looks right to me.
For what it's worth, I got sizes back down to what I expected by 
bumping

from my previous --modes=better down to --modes=good.

I think you've missed the point of the thread. People have been
bemoaning the loss of 1280 x720 25fps and the fact that if you want that
resolution you have to download the 50fps option which doubles the file
size and download time. Your mode produces a lower (SD rather than HD)
result.

Yes -- hence the "for what it's worth" -- people searching the archives
and being attracted by the "I really would like to get back to 1 hour >
approx 1GB" size issue may be as insensitive to the fine details as I am
and can take a similarly easy course as I, so I figure it will still be
worth something to some. Still, thank you for the useful clarification!
(The only time I care about the difference is when the download takes
ages over my semi-rural ADSL or my budget hardware can barely display
fast enough. I do notice the difference with HD but don't especially
need to see every wrinkle on politicians' faces.)


I agree with the point you make, but I repeat, previous posters in the 
thread were saying they wanted HD not SD, hence my post.


I'm actually in that group, the previous 25fps setup leading to 
1gig/hour of programming suited me but I recognise the goalposts have 
now moved. I'm in the fortunate position of living in Manchester and 
having a 200 Mbps service from Virgin, so download times aren't really 
an issue for me. I tend to download the previous day's programmes 
overnight and depending on the  CDN I can often get around 150-170 Mbps. 
Hard drive space is more of an issue for me. Given where we are, I have 
split my downloads into HD and SD (agreeing with the point you make re 
resolution of politicians' faces).


Even there, although 50fps is great for certain programmes, it's an 
overkill for others and I really would like the old 25fps HD option - to 
be honest I would prefer that as my default. Hence I'm keeping an eye on 
the discussion here as to the best way of reducing downloaded content 
from 50 down to 25fps.


Hi Alan and Jim,
Your summary of what I too want is correct; 1280x720 25fps.

I can download using --raw but please could somebody help with the 
command for getting this from the raw .ts file.


I do not need anything clever using ram or temporary files, just a 
simple command I can run on the raw download file to get me the .mp4 
file at 1280 x 720 x 25fps.  Either ffmpeg or Handbrake will do, 
whichever works!


Once I have this working I can build on it but right now I am lost in 
the technical stuff above on frames etc. which leaves me unsure if what 
I seek is possible.


Budge.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Alan Milewczyk]

On 20/05/2018 10:16, Jim web wrote:


And poking may make things worse. So end
of story, sadly.

Jim

Thanks, Jim. I agree with all the points you make, it's a strange result 
that they quote but I'd rather live with the current system than risk 
losing GIP. Mind you, I shudder at the prospect of future file sizes 
once we get into 4K! Anyway, I appreciate the discussions you've been 
having.


Best wishes

Alan


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Alan Milewczyk]

On 20/05/2018 09:56, Mark Carroll wrote:

On 20 May 2018, Alan Milewczyk wrote:


On 19/05/2018 22:27, Mark Carroll wrote:

On 17 May 2018, Budge wrote:


I have tried to follow this thread as I have just found my download of
The Bridge file size and resolution are less than before.  Upgraded to
3.14.0 and forced a new download and file size is more than twice the
first.  I really would like to get back to 1 hour > approx 1GB and the
advice from iz looks right to me.

For what it's worth, I got sizes back down to what I expected by bumping
from my previous --modes=better down to --modes=good.

I think you've missed the point of the thread. People have been
bemoaning the loss of 1280 x720 25fps and the fact that if you want that
resolution you have to download the 50fps option which doubles the file
size and download time. Your mode produces a lower (SD rather than HD)
result.

Yes -- hence the "for what it's worth" -- people searching the archives
and being attracted by the "I really would like to get back to 1 hour >
approx 1GB" size issue may be as insensitive to the fine details as I am
and can take a similarly easy course as I, so I figure it will still be
worth something to some. Still, thank you for the useful clarification!
(The only time I care about the difference is when the download takes
ages over my semi-rural ADSL or my budget hardware can barely display
fast enough. I do notice the difference with HD but don't especially
need to see every wrinkle on politicians' faces.)


I agree with the point you make, but I repeat, previous posters in the 
thread were saying they wanted HD not SD, hence my post.


I'm actually in that group, the previous 25fps setup leading to 
1gig/hour of programming suited me but I recognise the goalposts have 
now moved. I'm in the fortunate position of living in Manchester and 
having a 200 Mbps service from Virgin, so download times aren't really 
an issue for me. I tend to download the previous day's programmes 
overnight and depending on the  CDN I can often get around 150-170 Mbps. 
Hard drive space is more of an issue for me. Given where we are, I have 
split my downloads into HD and SD (agreeing with the point you make re 
resolution of politicians' faces).


Even there, although 50fps is great for certain programmes, it's an 
overkill for others and I really would like the old 25fps HD option - to 
be honest I would prefer that as my default. Hence I'm keeping an eye on 
the discussion here as to the best way of reducing downloaded content 
from 50 down to 25fps.


Regards

Alan




---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article <5f841aa5-a7dd-66f3-17ed-bc34c3dfd...@soulman1949.com>, Alan
Milewczyk  wrote:

> I think you've missed the point of the thread. People have been
> bemoaning the loss of 1280 x720 25fps and the fact that if you want that
> resolution you have to download the 50fps option which doubles the file
> size and download time. 

I've been trying to get more info on this from my 'usual channels'. So far
as I can tell, the situation was that the 25fps mode was a hangover from
the 'Red Bee' period. Apparently no-one had noticed that it was still
enabled... until a short while ago. When it was noticed, it was removed.

I've been lead to believe that some 'comparison tests' were done and it was
felt that the lower res modes 'looked better' than the 1280 x 720 25fps.
Which is quite an odd outcome. However I get the impression that in
reality, essentially the *only* users of the 25fps was actually gip
requests. Which means, officially, it wasn't 'in demand'. So it was chopped
rather than something else.

I have been trying to get though to someone involved in the tests/decision
to talk to me via others. But the snag is that the above means somehow
avoiding anyone 'noticing' that gip comes into it. This mode wasn't
'actually on offer' anyway. The risk is that it may draw attention to gip
and then an official line agin it. Game not worth candle.

I have tried making the points about the advantages of the 25fps, and
questioned the strange test results. But I fear this is as far as it goes,
and we won't be seeing the 25fps again. Personally, I think this is crazy
as I suspect it would actually reduce the loading on the BBC arrangements
(and hence costs) if the BBC *promoted* 25fps. Given a choice of half the
data rate I suspect many would opt for it. But it seems unlikely this idea
will cause a change of mind now. And poking may make things worse. So end
of story, sadly.

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Mark Carroll]

On 20 May 2018, Alan Milewczyk wrote:

> On 19/05/2018 22:27, Mark Carroll wrote:
>> On 17 May 2018, Budge wrote:
>>
>>> I have tried to follow this thread as I have just found my download of
>>> The Bridge file size and resolution are less than before.  Upgraded to
>>> 3.14.0 and forced a new download and file size is more than twice the
>>> first.  I really would like to get back to 1 hour > approx 1GB and the
>>> advice from iz looks right to me.
>> For what it's worth, I got sizes back down to what I expected by bumping
>> from my previous --modes=better down to --modes=good.
>
> I think you've missed the point of the thread. People have been 
> bemoaning the loss of 1280 x720 25fps and the fact that if you want that 
> resolution you have to download the 50fps option which doubles the file 
> size and download time. Your mode produces a lower (SD rather than HD) 
> result.

Yes -- hence the "for what it's worth" -- people searching the archives
and being attracted by the "I really would like to get back to 1 hour >
approx 1GB" size issue may be as insensitive to the fine details as I am
and can take a similarly easy course as I, so I figure it will still be
worth something to some. Still, thank you for the useful clarification!
(The only time I care about the difference is when the download takes
ages over my semi-rural ADSL or my budget hardware can barely display
fast enough. I do notice the difference with HD but don't especially
need to see every wrinkle on politicians' faces.)

-- Mark

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Alan Milewczyk]

On 19/05/2018 22:27, Mark Carroll wrote:

On 17 May 2018, Budge wrote:


I have tried to follow this thread as I have just found my download of
The Bridge file size and resolution are less than before.  Upgraded to
3.14.0 and forced a new download and file size is more than twice the
first.  I really would like to get back to 1 hour > approx 1GB and the
advice from iz looks right to me.

For what it's worth, I got sizes back down to what I expected by bumping
from my previous --modes=better down to --modes=good.

-- Mark


I think you've missed the point of the thread. People have been 
bemoaning the loss of 1280 x720 25fps and the fact that if you want that 
resolution you have to download the 50fps option which doubles the file 
size and download time. Your mode produces a lower (SD rather than HD) 
result.


A


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Mark Carroll]

On 17 May 2018, Budge wrote:

> I have tried to follow this thread as I have just found my download of 
> The Bridge file size and resolution are less than before.  Upgraded to 
> 3.14.0 and forced a new download and file size is more than twice the 
> first.  I really would like to get back to 1 hour > approx 1GB and the 
> advice from iz looks right to me.

For what it's worth, I got sizes back down to what I expected by bumping
from my previous --modes=better down to --modes=good.

-- Mark

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

RE: no more hslv format ?

[Andy Wedge]

> -Original Message-
> From: get_iplayer [mailto:get_iplayer-boun...@lists.infradead.org] On
> Behalf Of Budge
> Sent: 17 May 2018 23:39
> I have tried to follow this thread as I have just found my download of The
> Bridge file size and resolution are less than before.  Upgraded to
> 3.14.0 and forced a new download and file size is more than twice the first.  
> I
> really would like to get back to 1 hour > approx 1GB and the advice from iz
> looks right to me.
> 
> Can anybody help me with scripts to achieve what I seek please?
> Preferred operation would be to implement the download as cron job as
> now and then with a bunch of downloaded raw files in a directory run a script
> to give me my .mp4 files for loading onto NAS.
> 
> Budge

Hi Budge,

It sounds like you're getting the default HD streams which will give you a file 
size of just over 2GB for a 1 hour programme.

If you read https://github.com/get-iplayer/get_iplayer/wiki/modes and 
https://github.com/get-iplayer/get_iplayer/wiki/modesref#hlshd they give a very 
good explanation of the various combinations of options. I have no need for HD 
myself so have gone for --fps25 which effectively eliminates HD downloads and 
halves the file size.  I've also gone for --tvmode=better which reduces the 
resolution a bit and reduces the file size even more so I end up with about 
0.8GB for a 1 hour programme. 

If you want the same you can add those options to your options file with the 
command:

get_iplayer --prefs-add --tvmode=better --fps25

I'm on Windoze so I cannot help with cron settings I'm afraid.

Cheers,
Andy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Budge]

On 03/05/18 13:35, iz wrote:

You may be able to limit the whole process to only two file writes
with a little scripting and XML parsing. Use --raw with GiP  to save
the .ts file directly to drive or to drive via ram disk. The ram disk
probably wouldn't be of much use since I doubt your downloads are
limited by disk write speed. Also use --metadata to save programme
metadata in an XML file and --thumbnail to save the cover art in a JPG
file. Write a script to re-encode the file to 25fps MP4 and at the
same time add metadata tags with ffmpeg, using the XML file and JPG
file as input to construction of the ffmpeg command string. You'll
need ffmpeg 4.0 or higher to add cover art to MP4.


I have tried to follow this thread as I have just found my download of 
The Bridge file size and resolution are less than before.  Upgraded to 
3.14.0 and forced a new download and file size is more than twice the 
first.  I really would like to get back to 1 hour > approx 1GB and the 
advice from iz looks right to me.


Can anybody help me with scripts to achieve what I seek please? 
Preferred operation would be to implement the download as cron job as 
now and then with a bunch of downloaded raw files in a directory run a 
script to give me my .mp4 files for loading onto NAS.


Budge

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[RS]

On 07/05/18 17:00, Ralph Corderoy wrote:


I too have metered broadband, but have automated use of it begin at
00:03 using timeout(1) to ensure it is killed by my 08:00 cut-off time
if it didn't complete for some reason.  Have you considered something
similar, perhaps using get_iplayer's `--pvr-queue', etc., to add
programmes of interest, and then telling it to `run' some or all of the
queue items?


Hi Jim and Ralph

Are you sure metered broadband is the best deal for you?  What you said 
about unlimited overnight use reminded me of a contract I had with 
plusnet which was unlimited from midnight to 0800.  Out of curiosity I 
tried to find the prices to compare.  It wasn't easy.  Sky won't tell 
you unless you have an account, but I did find the archived prices for 
plusnet Essentials.


Interestingly for new contracts there are only two metered offers.  They 
are both for Sky fibre and are more expensive than unmetered offers.


Although the old metered contracts sound cheap the difference is that it 
was necessary to pay separately for the line rental.  According to BT's 
website its monthly line rental is now £18.99.  The Post Office is 
offering unlimited ADSL2+ for £17/month including line rental so it is 
effectively paying you £2/month to take its unlimited broadband.


Those prices suggest it may be a good idea to review your contract.

Best wishes
Richard



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Christopher Woods]

Curious - 25p @ 1/50th is the generally accepted 180° rule, unless they're 
doing weird stylistic stuff in post? Or does it all just look smooshy and 
motion is really badly defined?


Personally I wish people would shoot 50p and frame drop instead of shoot 
25p, I think it's more flexible.


(Making me feel old now. The last yoof soap I watched regularly was Byker 
Grove)


On 7 May 2018 18:14:40 Lucy Walker  wrote:


On 07/05/2018 17:09, Christopher Woods wrote:


Steve is right on the doubled technique, this is the filmic look you
see on documentaries and dramas. That source material will be likely
be captured as progressive frames, then upconverted to 50 fields per
second, 'progressive segmented frame' format (PsF) in the edit.



At least one "yoof soap" shoots 1080p25 shuttered to 1/50th - looks like
shit, but the head of cameras is a fucking moron. No more processing
than that

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer





___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Lucy Walker]

On 07/05/2018 17:09, Christopher Woods wrote:


Steve is right on the doubled technique, this is the filmic look you 
see on documentaries and dramas. That source material will be likely 
be captured as progressive frames, then upconverted to 50 fields per 
second, 'progressive segmented frame' format (PsF) in the edit.




At least one "yoof soap" shoots 1080p25 shuttered to 1/50th - looks like 
shit, but the head of cameras is a fucking moron. No more processing 
than that


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Jim,

> > Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> > `--output /tmp' and you should see its intermediate files, and the
> > final file, only appear there.  `--command' could then move that
> > final file to the SSD, or run a conversion command that writes to
> > the SSD and then removes the tmpfs input.
>
> the df command listed various 'locations'. These included
>
> tmpfs  815696   1352  814334   /run
> none   4078460  0 4078460  /run/shm
> none   102400   8 102392   /run/usr

/run/shm is half your RAM, as I'd expect.  I'd also expect /tmp to be
there.  What does `df -T /tmp' show?  Perhaps that's using SSD for all
temporary files and you might not want that given your concerns.

> Inside /run/usr I did find a directory for my user id and could use
> that as the user.

But it's deliberately constrained to be small.

> However I don't know what the /run/shm is for, so need to do some
> finding out...

POSIX's communication methods of shared memory and semaphores;
shm_overview(7), and sem_overview(7).  Again, not what you want.

> I did set up an explict ramdisc on another machine ages ago by adding
> a line to the /etc/fstab file. But that fixed the size to 256 MB,
> which in this context is tiny. System monitor says I have 8GB of ram.

That's the right idea.  Here, the system has /tmp be a tmpfs that is
half the RAM, like the `shm' one above.  They share one single half.
`systemctl status tmp.mount' is what creates it on some systems these
days, not an /etc/fstab entry.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Christopher Woods]

On 3 May 2018 21:38:23 RS  wrote:


On 03/05/18 12:18, Steve Dodd wrote:


Is it possible it depends on the source material? From the BBC quote
earlier it sounds like their "source" material is still mostly HD
interlaced, but perhaps some of their sources are all also
progressive, and those ones get doubled identical frames (which might
be a logical result of feeding 25p material into something that's
designed to interpolate interlaced fields into frames)?

The satellite broadcasts I have seen from the BBC and other broadcasters
are 1920x1080i25 for HD and 720x576i25 or 704x576i25 for SD.  Everything
that is broadcast is in an interlaced format at some stage, however it
is generated.  Some programmes are only produced for the iPlayer, so
they may be generated in a different way.



A handful of HD is still 1440x1080i25 :( the non-square pixel format is a 
great cheat for bandwidth saving. The chroma compression on current 
generation terrestrial broadcasting also makes everything look bad. Once 
you watch uncompressed HD-SDI you never want anything else :) Some SD is 
way below what we would accept as SD resolution as well, all in the name of 
cost saving.



Steve is right on the doubled technique, this is the filmic look you see on 
documentaries and dramas. That source material will be likely be captured 
as progressive frames, then upconverted to 50 fields per second, 
'progressive segmented frame' format (PsF) in the edit.


The same image is stored across every two fields that constitute one frame, 
meaning you're being shown one effective whole full resolution image, 25 
times a second.


With standard interlaced footage, you're being shown overlapping effective 
'images' at half-resolution, 50 times a second. Persistence of vision 
(well, nowadays, flat panel processing circuitry) bob deinterlaces the 
fields to produce 50 images per second.



However, TVs sometimes do this artificially by taking true progressive 
sources and interpolating the footage to a higher refresh rate (100/200 
Hz). This inherently looks dire and should be disabled.


On some TVs with poorly written image DSP (or badly chosen settings) you'll 
see the picture suddenly go 'filmic' as the deinterlace method changes to 
blend, which literally blends together pairs of fields to produce 25 frames.


Some explainers of fields, frames and progressive segmented format interlaced:

https://en.m.wikipedia.org/wiki/Progressive_segmented_frame

https://wolfcrow.com/blog/understanding-terminology-progressive-segmented-frames-psf/
https://wolfcrow.com/blog/understanding-terminology-progressive-frames-interlaced-frames-and-the-field-rate/



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Jim,

> My concern is that I only have a total of 8GB of ram at present on the
> machine, and the 1280x720 50fps files tend to come in at 2GB or more
> per hour. 

Yes, you'd need to finish one download off, clearing RAM disk, before
starting the next.

> To save time and avoid running past 9am (and entering metered time) it
> would be quickest to do all the fetches first.

I too have metered broadband, but have automated use of it begin at
00:03 using timeout(1) to ensure it is killed by my 08:00 cut-off time
if it didn't complete for some reason.  Have you considered something
similar, perhaps using get_iplayer's `--pvr-queue', etc., to add
programmes of interest, and then telling it to `run' some or all of the
queue items?

> but have about three times as much data written to hd at some point.

Yes, with my preferred choices, get_iplayer's writes to disk would be

downloaded video
downloaded audio
video and audio combined into MP4
tagged MP4

so 3(v+a), and I'm not converting to 25 fps.

> I could use spinning rust I guess. but that seems messy.

Do you have it already available in the machine to use?  That seems an
easy compromise.

You could also look at your SSD's specifications and whether you're
striving too hard to avoid what might not be a problem with modern ones
and their built-in wear levelling.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Richard,

> > ffmpeg can produce PNGs, one per frame, and convert only a few
> > specific seconds to avoid tens of thousands of them
>
> Would you first need to convert the H.264 or H.265 to raw video?  If
> one PNG is of an I-frame and the next is a P-frame or B-frame they are
> bound to be different.

No, AIUI ffmpeg(1) produces a PNG that's the whole assembled picture,
just as if one hit `Pause' when watching TV.  Any indication of how that
was built up from the input video encoding is lost.

ffmpeg -i foo.mp4 -t 1 foo-%04d.png

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Dave Lambley]

On 5 May 2018 at 00:29, RS  wrote:
> On 04/05/18 23:50, Dave Lambley wrote:
>
>>
>> They would remove the interlace to bring the framerate to 50fps before
>> encoding. There are many methods of doing so, ffmpeg can do this for
>> example.
>>
> This article
>
> https://superuser.com/questions/253691/how-to-convert-108050i-72050p-using-ffmpeg
>
> says, "Just like interlacing progressive content divides each frame into 2
> fields, thus doubling the frame rate, the common method of deinterlacing is
> to combine each 2 fields into 1 frame, which reduces the rate by 2, thus
> taking 50 into 25fps. You can of course double each final frame, but that
> does not provide any benefit."
>
> Do you disagree?

eg., https://ffmpeg.org/ffmpeg-filters.html#bwdif (and other) filters
can give you good quality 50fps frames, without duplication, from
interlaced 25fps video. I have no idea what the BBC use, but they have
plenty of choice.

Dave

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article

Re: no more hslv format ?

[RS]

On 04/05/18 23:50, Dave Lambley wrote:



They would remove the interlace to bring the framerate to 50fps before
encoding. There are many methods of doing so, ffmpeg can do this for
example.


This article

https://superuser.com/questions/253691/how-to-convert-108050i-72050p-using-ffmpeg

says, "Just like interlacing progressive content divides each frame into 
2 fields, thus doubling the frame rate, the common method of 
deinterlacing is to combine each 2 fields into 1 frame, which reduces 
the rate by 2, thus taking 50 into 25fps. You can of course double each 
final frame, but that does not provide any benefit."


Do you disagree?

Best wishes
Richard



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Dave Lambley]

On 3 May 2018 at 00:24, RS  wrote:
> On 02/05/18 23:21, Owen Smith wrote:
>
>>
>> I've been mystified for a while why people talked about "dropping every
>> other frame" as if it were trivial to do, and an email earlier in this chain
>> looked like someone was trying to do that again. I was explaining why that
>> simply is not possible in the general case.
>>
>
> If I have caused confusion by talking about dropping alternate frames I
> apologise.  I had come across some posts in another forum which suggested it
> could be done, but I now recognise I was wrong.  What makes it worse is that
> I have since come across a thread in this listserver from two years ago
> where I was asking exactly the same questions, and Vangelis pointed out I
> was wrong and directed me to a Wikipedia article on H.264.
>
> It is not possible to change the frame rate using -c:v=copy in ffmpeg; it is
> necessary to re-encode which is why it takes so long.  Inevitably there will
> be losses, added to which the codecs available to us may be inferior to
> those used by the BBC.
>
> I think it was Nick Payne who said he had experimented with re-encoding in
> HEVC (H.265) and found that the file size was the same for 25fps as it was
> for 50fps, which led him to conclude that frames were being duplicated to
> achieve 50fps.
>
> The broadcast signal (at least on satellite) is 1920x1080i at 25fps usually
> with two audio streams, AC3 and NAR.  If the broadcast signal is recorded,
> the resultant file size is about 3GByte/h.
>
> The BBC'S explanation of what it does with the broadcast signal is, "The
> Elemental encoders are used to convert the 1920x1080 interlaced content to
> 960x540 for progressive encoding at 50fps."

Searching turns up this,

http://dpp-assets.s3.amazonaws.com/wp-content/uploads/specs/bbc/TechnicalDeliveryStandardsBBCFile.pdf

... which tells you that 25fps interlaced is what programme makers
will be supplying for HD programmes.

> It also says, "The 50fps, 1280x720 profile, however, will be available to
> those with 5Mbit/s broadband connections." but it does not explain where it
> comes from or why it cannot generate a 1280x720p 25fps profile.

They would remove the interlace to bring the framerate to 50fps before
encoding. There are many methods of doing so, ffmpeg can do this for
example.

Dave

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article
, iz
 wrote:
> On 2 May 2018 at 16:49, Jim web  wrote:
> >
> > As things are, I could then have gip+ffmpeg generate the 50 fps mp4
> > files (i.e. in the current form) onto hd. This saves some hd wear but
> > means I now have those (big) files on hd before I then run a 50 -> 25
> > fps process to create smaller ones (which will be akin in size to what
> > I got from the 25fps fetching). The result using this approach means
> > I've still done the fetching as quickly as possible, but have about
> > three times as much data written to hd at some point.

> You may be able to limit the whole process to only two file writes with
> a little scripting and XML parsing. 

FWIW as things stand I run the processes via a simple proglet I wrote in
'C'. (I use it as a ROX-Filer app, but it can also work via CLI.) I can
drag-and-drop a text file onto the program's icon. The file specifies a
'mode' and a pid per line, and the program then proceeds to chug though
them as I make breakfast. If needed, the file I give it could also say if
any particular reprocessing need be done 'now' or left to later, etc.

So I'd almost certainly do this via a 'C' approach because I've become used
to 'C' over the years. The trick is knowing what commands to launch, etc.

Modifying where output goes, making the output -raw, etc, is OK as I know
how to do that - once I've sorted out unknowns like if the -raw is going to
ramdisc or somewhere else, when reprocessing is done, etc.

Thus, yes, I am thinking it should be possible to do this via running 'one
program'. But...

Truth is, having finally allowed people to prise FORTRAN out of my cold
dead hands a decade or more ago, I'm now settled on 'C'. :-)

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[RS]

On 03/05/18 12:18, Steve Dodd wrote:


Is it possible it depends on the source material? From the BBC quote
earlier it sounds like their "source" material is still mostly HD
interlaced, but perhaps some of their sources are all also
progressive, and those ones get doubled identical frames (which might
be a logical result of feeding 25p material into something that's
designed to interpolate interlaced fields into frames)?

The satellite broadcasts I have seen from the BBC and other broadcasters 
are 1920x1080i25 for HD and 720x576i25 or 704x576i25 for SD.  Everything 
that is broadcast is in an interlaced format at some stage, however it 
is generated.  Some programmes are only produced for the iPlayer, so 
they may be generated in a different way.


Best wishes
Richard



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[RS]

On 03/05/18 11:41, Ralph Corderoy wrote:


ffmpeg can produce PNGs, one per frame, and convert only a few specific
seconds to avoid tens of thousands of them, and then adjacent frames
could be compared in pairs to see if they are indeed identical;  compare
`ab', `bc', `cd'...

There's a couple of variants for comparing images.

 compare foo.png bar.png x:
 gm compare -highlight-style assign foo.png bar.png -file x:


Hi Ralph

Would you first need to convert the H.264 or H.265 to raw video?  If one 
PNG is of an I-frame and the next is a P-frame or B-frame they are bound 
to be different.


Best wishes
Richard


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[iz]

On 2 May 2018 at 16:49, Jim web  wrote:
>
> As things are, I could then have gip+ffmpeg generate the 50 fps mp4 files
> (i.e. in the current form) onto hd. This saves some hd wear but means I now
> have those (big) files on hd before I then run a 50 -> 25 fps process to
> create smaller ones (which will be akin in size to what I got from the
> 25fps fetching). The result using this approach means I've still done the
> fetching as quickly as possible, but have about three times as much data
> written to hd at some point.

You may be able to limit the whole process to only two file writes
with a little scripting and XML parsing. Use --raw with GiP  to save
the .ts file directly to drive or to drive via ram disk. The ram disk
probably wouldn't be of much use since I doubt your downloads are
limited by disk write speed. Also use --metadata to save programme
metadata in an XML file and --thumbnail to save the cover art in a JPG
file. Write a script to re-encode the file to 25fps MP4 and at the
same time add metadata tags with ffmpeg, using the XML file and JPG
file as input to construction of the ffmpeg command string. You'll
need ffmpeg 4.0 or higher to add cover art to MP4.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Steve Dodd]

On Thu, May 3, 2018 at 12:24 AM, RS  wrote:

> I think it was Nick Payne who said he had experimented with re-encoding in 
> HEVC (H.265) and found that the file size was the same for 25fps as it was 
> for 50fps, which led him to conclude that frames were being duplicated to 
> achieve 50fps.

Is it possible it depends on the source material? From the BBC quote
earlier it sounds like their "source" material is still mostly HD
interlaced, but perhaps some of their sources are all also
progressive, and those ones get doubled identical frames (which might
be a logical result of feeding 25p material into something that's
designed to interpolate interlaced fields into frames)?

S.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Richard,

> I think it was Nick Payne who said he had experimented with
> re-encoding in HEVC (H.265) and found that the file size was the same
> for 25fps as it was for 50fps, which led him to conclude that frames
> were being duplicated to achieve 50fps.

ffmpeg can produce PNGs, one per frame, and convert only a few specific
seconds to avoid tens of thousands of them, and then adjacent frames
could be compared in pairs to see if they are indeed identical;  compare
`ab', `bc', `cd'...

There's a couple of variants for comparing images.

compare foo.png bar.png x:
gm compare -highlight-style assign foo.png bar.png -file x:

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Nick,

> I had some of the coverage of the UK snooker championships
...
> 4h58m match: D/L size 5.03Gb @ 25fps, output from Handbrake was 1.55Gb
> 4h44m match: D/L size 10.2Gb @ 50fps, output from Handbrake was 1.48Gb

Perhaps many frames(!) of snooker coverage isn't ideal for this
comparison as it often is a static picture of the table.  :-)

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article <20180502152015.50ea621...@orac.inputplus.co.uk>, Ralph
Corderoy  wrote:
> > The challenge for me is to work out how to get the fetched file to go
> > onto the tmpfs

> Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> `--output /tmp' and you should see its intermediate files, and the final
> file, only appear there.  `--command' could then move that final file to
> the SSD, or run a conversion command that writes to the SSD and then
> removes the tmpfs input.

the df command listed various 'locations'. These included

tmpfs  815696  1352  814334  /run

none 4078460  0  4078460  /run/shm

none 102400  8  102392  /run/usr

Inside /run/usr I did find a directory for my user id and could use that as
the user. However I don't know what the /run/shm is for, so need to do some
finding out...

I did set up an explict ramdisc on another machine ages ago by adding a
line to the /etc/fstab file. But that fixed the size to 256 MB, which in
this context is tiny. System monitor says I have 8GB of ram.

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article <20180502152015.50ea621...@orac.inputplus.co.uk>, Ralph
Corderoy  wrote:
> >
> > The challenge for me is to work out how to get the fetched file to go
> > onto the tmpfs

> Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> `--output /tmp' and you should see its intermediate files, and the final
> file, only appear there.  `--command' could then move that final file to
> the SSD, or run a conversion command that writes to the SSD and then
> removes the tmpfs input.

Thanks. :-)

> As for altering the ffmpeg command that get_iplayer is using, I'm not
> sure that's worthwhile?  It isn't doing any transcoding, just changing
> the container format, or splicing in better audio, that kind of thing.
> So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second
> lossy one that you'd prefer to combine with the first.  I could be
> wrong, not knowing how to have ffmpeg do this conversion.  When you find
> out, let the list know.  :-)

My concern is that I only have a total of 8GB of ram at present on the
machine, and the 1280x720 50fps files tend to come in at 2GB or more per
hour. 

To save time and avoid running past 9am (and entering metered time) it
would be quickest to do all the fetches first. But this will mean more
files that I can store on ram on some days. So the wish would be to then
shift or process material and put it on hd.

As things are, I could then have gip+ffmpeg generate the 50 fps mp4 files
(i.e. in the current form) onto hd. This saves some hd wear but means I now
have those (big) files on hd before I then run a 50 -> 25 fps process to
create smaller ones (which will be akin in size to what I got from the
25fps fetching). The result using this approach means I've still done the
fetching as quickly as possible, but have about three times as much data
written to hd at some point.

It would be nice to do the process file by file and in each case only the
'final' 25 fps version gets written to hd. This means the same hd use as
previously. But that either means I need enough ram to hold all the 'temp'
files (masses of ram needed) or doing the process item by item. (slow)

I could use spinning rust I guess. but that seems messy. Otherwise it seems
to want me to buy and fit a lot of ram. :-)

So at present I'm wondering and experimenting. 8-]

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Nick Payne]

On 3/05/2018 9:24 AM, RS wrote:
> On 02/05/18 23:21, Owen Smith wrote:
>
>>
>> I've been mystified for a while why people talked about "dropping
>> every other frame" as if it were trivial to do, and an email earlier
>> in this chain looked like someone was trying to do that again. I was
>> explaining why that simply is not possible in the general case.
>>
>
> If I have caused confusion by talking about dropping alternate frames
> I apologise.  I had come across some posts in another forum which
> suggested it could be done, but I now recognise I was wrong.  What
> makes it worse is that I have since come across a thread in this
> listserver from two years ago where I was asking exactly the same
> questions, and Vangelis pointed out I was wrong and directed me to a
> Wikipedia article on H.264.
>
> It is not possible to change the frame rate using -c:v=copy in ffmpeg;
> it is necessary to re-encode which is why it takes so long. 
> Inevitably there will be losses, added to which the codecs available
> to us may be inferior to those used by the BBC.
>
> I think it was Nick Payne who said he had experimented with
> re-encoding in HEVC (H.265) and found that the file size was the same
> for 25fps as it was for 50fps, which led him to conclude that frames
> were being duplicated to achieve 50fps.

Yes, I had some of the coverage of the UK snooker championships where
they provided HLS downloads @ 1280x720 25fps and other coverage was only
available as HVF downloads @ 1280x720 50fps. When I ran these both
through Handbrake with identical settings to convert to HEVC, retaining
the frame rate of the downloaded files, the size reduction for the 50fps
downloads was about twice that for the 25fps downloads, and the size of
the files output by Handbrake was proportional to the length of the
program and not the frame rate. e.g.

4h58m match: D/L size 5.03Gb @ 25fps, output from Handbrake was 1.55Gb
4h44m match: D/L size 10.2Gb @ 50fps, output from Handbrake was 1.48Gb

Nick


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[RS]

On 02/05/18 23:21, Owen Smith wrote:



I've been mystified for a while why people talked about "dropping every other 
frame" as if it were trivial to do, and an email earlier in this chain looked like 
someone was trying to do that again. I was explaining why that simply is not possible in 
the general case.



If I have caused confusion by talking about dropping alternate frames I 
apologise.  I had come across some posts in another forum which 
suggested it could be done, but I now recognise I was wrong.  What makes 
it worse is that I have since come across a thread in this listserver 
from two years ago where I was asking exactly the same questions, and 
Vangelis pointed out I was wrong and directed me to a Wikipedia article 
on H.264.


It is not possible to change the frame rate using -c:v=copy in ffmpeg; 
it is necessary to re-encode which is why it takes so long.  Inevitably 
there will be losses, added to which the codecs available to us may be 
inferior to those used by the BBC.


I think it was Nick Payne who said he had experimented with re-encoding 
in HEVC (H.265) and found that the file size was the same for 25fps as 
it was for 50fps, which led him to conclude that frames were being 
duplicated to achieve 50fps.


The broadcast signal (at least on satellite) is 1920x1080i at 25fps 
usually with two audio streams, AC3 and NAR.  If the broadcast signal is 
recorded, the resultant file size is about 3GByte/h.


The BBC'S explanation of what it does with the broadcast signal is, "The 
Elemental encoders are used to convert the 1920x1080 interlaced content 
to 960x540 for progressive encoding at 50fps."


It also says, "The 50fps, 1280x720 profile, however, will be available 
to those with 5Mbit/s broadband connections." but it does not explain 
where it comes from or why it cannot generate a 1280x720p 25fps profile.


Best wishes
Richard


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Owen Smith]

I'm replying on an iPad, which makes large scale editing of email replies a 
huge amount of work. So I top post, because it takes only a tenth of the time. 
If Apple made it only cost me double the amount of time to reply properly I'd 
do it.

I've been mystified for a while why people talked about "dropping every other 
frame" as if it were trivial to do, and an email earlier in this chain looked 
like someone was trying to do that again. I was explaining why that simply is 
not possible in the general case.

-- 
Owen Smith 
Cambridge, UK

> On 2 May 2018, at 22:33, Ralph Corderoy  wrote:
> 
> Hi Owen,
> 
>> What do you mean this isn't a lossy transcoding?
> 
> Is that aimed at me?
> 
> Perhaps if you didn't top post, and instead wrote that under a quote of
> mine I'd know to which bit of the two ffmpeg invocations you were
> referring!  :-)
>> How can ffmpeg go from 50fps to 25fps without losing anything?
> 
> I don't think it can, and didn't suggest it could.  It is lossy.  I said
> the first, default, one wasn't, and that therefore it wasn't worth
> combining this extra, 50->25, one with it, as you would want to if both
> were lossy.
> 
> -- 
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy
> 
> ___
> get_iplayer mailing list
> get_iplayer@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Owen,

> What do you mean this isn't a lossy transcoding?

Is that aimed at me?

Perhaps if you didn't top post, and instead wrote that under a quote of
mine I'd know to which bit of the two ffmpeg invocations you were
referring!  :-)

> How can ffmpeg go from 50fps to 25fps without losing anything?

I don't think it can, and didn't suggest it could.  It is lossy.  I said
the first, default, one wasn't, and that therefore it wasn't worth
combining this extra, 50->25, one with it, as you would want to if both
were lossy.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Owen Smith]

That doesn't make any difference to the H.264 encoding. At best if you get 
lucky every odd frame will be marked as being interpolated from the preceding 
even frame and then no actual changes within the frame. But all it takes is for 
the encoder to slip by a frame somewhere or to encode an I frame (a complete 
standalone frame) in the even or odd half that you are trying to discard every 
one of and then you are screwed.

And if the 50fps file genuinely does have every other frame saying "I'm 
identical to the previous frame" the stripping these won't save much on the 
file size. The fact that people complain the 50fps files are double the size of 
the same resolution 25fps files implies to me this isn't how it has been done. 
Even if the BBC have only created motion interpolated frames from a 25fps 
source, they are still encoded in the video stream and you still can't casually 
toss alternate frames without screwing up the I-P-B frame interpolation.

-- 
Owen Smith 
Cambridge, UK

> On 2 May 2018, at 19:43, Peter S Kirk  wrote:
> 
> Many posts back it was mentioned they are not true 50fps, instead each 
> frame from a 25fps is duplicated merely to allow BBC to boast about 50fps 
> streaming.
> 
> On 2 May 2018 at 19:07, Owen Smith Owen Smith  
> wrote:
> 
>> What do you mean this isn't a lossy transcoding? How can ffmpeg go from 
>> 50fps to 25fps without losing anything? The
>> frames are not all complete frames, software can't just throw alternate 
>> frames away. Well it could,  but the only way
>> to do that is a full H.264 decode, then discard alternate frames, then a 
>> full H.264 encode again which is going to
>> involve loss.
>> 
>> Most frames are not fully present in the original stream, they are 
>> interpolated from previous and subsequent frames.
>> You can't throw any of those away, because other frames are interpolated 
>> from them. It would need to be a very special
>> original encode which had all even frames only interpolated from other even 
>> frames and ditto for odd frames to allow
>> alternate frames to be discarded. And a special encode like that would bloat 
>> the file size substantially, almost
>> doubling it I would expect.
> 
> 
> 
> ___
> get_iplayer mailing list
> get_iplayer@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Peter S Kirk]

Many posts back it was mentioned they are not true 50fps, instead each 
frame from a 25fps is duplicated merely to allow BBC to boast about 50fps 
streaming.

On 2 May 2018 at 19:07, Owen Smith Owen Smith  
wrote:

> What do you mean this isn't a lossy transcoding? How can ffmpeg go from 50fps 
> to 25fps without losing anything? The
> frames are not all complete frames, software can't just throw alternate 
> frames away. Well it could,  but the only way
> to do that is a full H.264 decode, then discard alternate frames, then a full 
> H.264 encode again which is going to
> involve loss.
> 
> Most frames are not fully present in the original stream, they are 
> interpolated from previous and subsequent frames.
> You can't throw any of those away, because other frames are interpolated from 
> them. It would need to be a very special
> original encode which had all even frames only interpolated from other even 
> frames and ditto for odd frames to allow
> alternate frames to be discarded. And a special encode like that would bloat 
> the file size substantially, almost
> doubling it I would expect.



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Owen Smith]

What do you mean this isn't a lossy transcoding? How can ffmpeg go from 50fps 
to 25fps without losing anything? The frames are not all complete frames, 
software can't just throw alternate frames away. Well it could,  but the only 
way to do that is a full H.264 decode, then discard alternate frames, then a 
full H.264 encode again which is going to involve loss.

Most frames are not fully present in the original stream, they are interpolated 
from previous and subsequent frames. You can't throw any of those away, because 
other frames are interpolated from them. It would need to be a very special 
original encode which had all even frames only interpolated from other even 
frames and ditto for odd frames to allow alternate frames to be discarded. And 
a special encode like that would bloat the file size substantially, almost 
doubling it I would expect.

-- 
Owen Smith 
Cambridge, UK

> On 2 May 2018, at 16:20, Ralph Corderoy  wrote:
> 
> Hi Jim,
> 
>>> You could use get_iplayer's --command option to run a command to
>>> move each final file off tmpfs as the download is finished.  Its
>>> --output affects all the intermediate files too, AIUI.
>> 
>> The challenge for me is to work out how to get the fetched file to go
>> onto the tmpfs
> 
> Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> `--output /tmp' and you should see its intermediate files, and the final
> file, only appear there.  `--command' could then move that final file to
> the SSD, or run a conversion command that writes to the SSD and then
> removes the tmpfs input.
> 
> As for altering the ffmpeg command that get_iplayer is using, I'm not
> sure that's worthwhile?  It isn't doing any transcoding, just changing
> the container format, or splicing in better audio, that kind of thing.
> So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second
> lossy one that you'd prefer to combine with the first.  I could be
> wrong, not knowing how to have ffmpeg do this conversion.  When you find
> out, let the list know.  :-)
> 
> -- 
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy
> 
> ___
> get_iplayer mailing list
> get_iplayer@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Jim,

> > You could use get_iplayer's --command option to run a command to
> > move each final file off tmpfs as the download is finished.  Its
> > --output affects all the intermediate files too, AIUI.
>
> The challenge for me is to work out how to get the fetched file to go
> onto the tmpfs

Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
`--output /tmp' and you should see its intermediate files, and the final
file, only appear there.  `--command' could then move that final file to
the SSD, or run a conversion command that writes to the SSD and then
removes the tmpfs input.

As for altering the ffmpeg command that get_iplayer is using, I'm not
sure that's worthwhile?  It isn't doing any transcoding, just changing
the container format, or splicing in better audio, that kind of thing.
So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second
lossy one that you'd prefer to combine with the first.  I could be
wrong, not knowing how to have ffmpeg do this conversion.  When you find
out, let the list know.  :-)

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[RS]

On 30/04/18 13:56, Jim web wrote:

I've been discussing the 'loss' of the 1280x720 25fps version with someone
at the BBC.

IIUC this stemmed from 'Red Bee' days of yore, and until recently people at
the BBC had thought they had stopped it long ago. Someone apparently
noticed recently that it was still available. And then actually disabled
it. Hence the mysterious recent ending of its availablity.

I have made the point that the 1280x720 25fps version is useful for people
with a content 'cap' problem, etc, so will be missed because the 50fps
version means somewhat bigger files and/or stream rates. But I doubt this
will cause a rethink.

We have known for some time that the BBC was going to stop using Flash 
at some stage.  Dinky anticipated that change and get_iplayer no longer 
supports Flash.  The last version with Flash capability was v2.99.


HLS is a much more recent innovation than Flash.  get_iplayer now has 
its own built-in downloader which downloads HLS 2 or 3 times as fast as 
Flash.  HLS has been referred to as a legacy mode so it was always 
likely that it would be removed eventually.  The surprise was that HLS 
was removed at the same time as Flash.


It is unrealistic to expect the BBC to restore HLS or Flash.

I have not seen any comments here which suggest that anyone is unhappy 
with HVF.  get_iplayer uses the same built-in downloader as for HLS. 
The BBC's full list of HVF modes is set out in a table in this document.


https://www.bbc.co.uk/rd/blog/2015-07-the-development-of-new-video-factory-profiles-for-bbc-iplayer

What I think a number of people here would like to see is the addition 
of one more High H.264 profile mode to that list, namely 1280x720p 25fps.


The BBC refers to viewing tests it has carried out, "using a range of 
content and included clips, from popular shows such as Strictly Come 
Dancing, East Enders and Top Gear."


It concludes, "At around 3 Mbit/s a 960x540 profile at 50fps will be 
made available to Connected TVs and set top boxes. The Elemental 
encoders are used to convert the 1920x1080 interlaced content to 960x540 
for progressive encoding at 50fps. Although the 960x540 profile has a 
reduced spatial resolution compared to the outgoing 1280x720 at 25fps, 
subjective assessments shows it delivers significantly better pictures 
on TV screens across a wide range of popular content (such as EastEnders 
and Top Gear) due to its higher frame rate. The 50fps, 1280x720 profile, 
however, will be available to those with 5Mbit/s broadband connections.


"Additional lower bit-rate profiles will be made available to computers 
on wifi. This enables video playback to continue, even when an 
individual's available bit-rate is reduced by users sharing a 
connection. These profiles will also cater for similar bit-rate 
restrictions on public wifi connections."


I accept that there is a trade off between resolution and frame rate. 
With differential encoding schemes like H.264, at a higher frame rate 
the changes between frames are smaller, so there is less to encode. 
Even so, for the uncompressed video the resolution at 50fps would need 
to be reduced to 905x510p to give the same bit rate as 1280x720p at 25fps.


For that reason I find it surprising that 960x540p at 50fps "delivers 
significantly better pictures on TV screens across a wide range of 
popular content (such as EastEnders and Top Gear) due to its higher 
frame rate" than 1280x720p at 25fps but the BBC has done the viewing 
tests and I have not.


More importantly BBC Four and to a lesser extent BBC 2 have a lot of 
programmes about paintings, sculpture, architecture and nature where 
there is a lot of fine detail and little motion.  Intuitively such 
programmes would not benefit from the higher frame rate but would 
benefit from the higher resolution.  There is no mention in the blog 
that such programmes were included in the viewing tests, and they ought 
to have been.


Best wishes
Richard




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Steven Carr]

On 1 May 2018 at 14:05, Jim web  wrote:
> I'll look at what the --verbose gives me as per your other email. Someone
> elsewhere has said they can get handbrake to do 50 -> 25 fps fairly well.
> But that means running that as well unless I can work out what it is doing
> on a command level. And IIUC handrake is using ffmpeg anyway under the
> hood...

I've just done something similar.

I get get_iplayer to download the files (tvbest with 50fps) and then
dump them into a folder, I then have another script that's going
through those files in that folder and converting them to x265 and
reducing the fps to 25. No issues so far, and results in a filesize of
around 350MB/per hour of TV.

It's using HandbrakeCLI (and fwiw Handbrake doesn't always use ffmpeg,
it also has other encoders built in), I'm encoding to x265 for the
smaller filesize but you can tell it to use x264 instead.

Only downside is the x265 encoding takes a while, pretty much double
the length of the video time on my system, about 1hr45 encoding for a
50-60 minute video, but hey ho, it works, and it's running in the
background via cron so it will spit out the videos when done - it's
not like I was in a hurry to watch them anyway...

For anyone interested the script is here...
https://pastebin.com/cGSxbSPK

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article <20180501113138.870f721...@orac.inputplus.co.uk>, Ralph
Corderoy  wrote:
> Hi Jim,

> > I've been discussing the 'loss' of the 1280x720 25fps version with
> > someone at the BBC.

> I miss those 1 GiB ~= 1 hour ones too.  They were `just right'.

Yes. It seems bonkers to me for the BBC to end them. But I've not had any
reaction to pointing out that they are useful, etc. So I assume they won't
be re-instated. :-/

> > It has also set me wondering about arranging for gip to fetch to ram
> > storage and then convert that into a file on my main disc.

> Is this Linux?  get_iplayer here uses the current working directory for
> all its large intermediate files.  If that was a `tmpfs' filesystem then
> they would all sit in RAM, unless you've swap space and pressure caused
> pages from the filesystem to be swapped out.

> You could use get_iplayer's --command option to run a command to move
> each final file off tmpfs as the download is finished.  Its --output
> affects all the intermediate files too, AIUI.

Yes, Linux. The challenge for me is to work out how to get the fetched file
to go onto the tmpfs, then have that processed - ideally having gip give
ffmpeg relevant details - into a 25fps file on my main disc. i.e. I don't
know enough to specify the commands and either get gip to do this, or add
in the relevant ffmpeg call.

Am I right that by using the --command and also --raw I can then run ffmpeg
to do the conversion and put the result where I want? That sounds about
right for what I have in mind. 

I'll look at what the --verbose gives me as per your other email. Someone
elsewhere has said they can get handbrake to do 50 -> 25 fps fairly well.
But that means running that as well unless I can work out what it is doing
on a command level. And IIUC handrake is using ffmpeg anyway under the
hood...

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Alan Milewczyk]

On 01/05/2018 12:31, Ralph Corderoy wrote:

Hi Jim,


I've been discussing the 'loss' of the 1280x720 25fps version with
someone at the BBC.

I miss those 1 GiB ~= 1 hour ones too.  They were `just right'.

Me too! :-(



It has also set me wondering about arranging for gip to fetch to ram
storage and then convert that into a file on my main disc.

Is this Linux?  get_iplayer here uses the current working directory for
all its large intermediate files.  If that was a `tmpfs' filesystem then
they would all sit in RAM, unless you've swap space and pressure caused
pages from the filesystem to be swapped out.


I'm using Windows 7 not Linux but I wouldn't have a clue how to do that.

You could use get_iplayer's --command option to run a command to move
each final file off tmpfs as the download is finished.  Its --output
affects all the intermediate files too, AIUI.

I've run the suggested command line instruction posted earlier and I'm 
afraid it's a very time consuming process - assuming nothing else is 
going on on my PC I get a conversion speed of approx 1.5x/1.6x (down to 
maybe 1.2x if other processes are active). Maybe some of that could be 
due to the relative slowness of hard drives. It would be good to know 
how to modify the code to convert the programme files on the fly.



Regards


Alan


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Jim,

> How do I find out the command gip sends to ffmpeg to do its default
> conversion as things stand?

Try `--verbose';  that should print external commands that are run.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Jim,

> I've been discussing the 'loss' of the 1280x720 25fps version with
> someone at the BBC.

I miss those 1 GiB ~= 1 hour ones too.  They were `just right'.

> It has also set me wondering about arranging for gip to fetch to ram
> storage and then convert that into a file on my main disc.

Is this Linux?  get_iplayer here uses the current working directory for
all its large intermediate files.  If that was a `tmpfs' filesystem then
they would all sit in RAM, unless you've swap space and pressure caused
pages from the filesystem to be swapped out.

You could use get_iplayer's --command option to run a command to move
each final file off tmpfs as the download is finished.  Its --output
affects all the intermediate files too, AIUI.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

My ISP is Orpheus internet. They aren't cheap, but they are excellent in
terms of service, etc. [1] 

I think my actual connection is a repackage of a BT/OR one that has a cap
at a rolling 50GB/30days. But only the time betweem 9am and midnight is
metered. So I can 'fill my boots' outwith that time if I wish. I then
typically get around 50-60 meg download rates since we got FTTC.

Jim

[1] They helpfully support a range of OS's including my actual favourite,
RISC OS. To give an idea of the service, the owner will give clients his
mobile number in case they need help out-of-hours. I once did at the
weekend, phoned him, and he was out sailing. He said he'd fix the problem,
and it was fixed a short time later.

In article <5ae779a9.20973.a9d9...@peter.kirk.isauk.biz>, Peter S Kirk
 wrote:
> Jim,

> What ISP and plan is that?

> On 30 Apr 2018 at 13:56, Jim web Jim web  wrote:

> > I can avoid a 'cap' as I do all the fetching before the 9am start of
> > our 'metered' period. 



> ___ get_iplayer mailing list
> get_iplayer@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Peter S Kirk]

Jim,

What ISP and plan is that?

On 30 Apr 2018 at 13:56, Jim web Jim web  wrote:

> I can avoid a 'cap' as I do all the fetching before the 9am start of our
> 'metered' period. 



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[iz]

On 30 April 2018 at 15:22, Jim web  wrote:
> conversion as things stand? IIRC it has to recontain the content and tweak
> the aac format in some way. So I'd need to be able to replilicate what is

GiP uses ffmpeg's aac_adtstoasc bitstream filter to remove ADTS
framing when copying the audio stream from .ts files, but more recent
versions of ffmpeg do that automatically for MP4 output format. If you
re-encode the audio as well as the video (probably not necessary),
that won't matter.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article
, iz
 wrote:
> On 30 April 2018 at 13:56, Jim web  wrote:
> >
> > 1) Use gip to get a 50fps file saved to ram.

> You'll want to use the --raw flag with GiP to avoid remuxing to MP4 and
> tagging, both of which create a file copy, which would halve the max
> size of programmes you could download to ram disk.

Yes. noted OK. :-) Following on from that, though...

How do I find out the command gip sends to ffmpeg to do its default
conversion as things stand? IIRC it has to recontain the content and tweak
the aac format in some way. So I'd need to be able to replilicate what is
needed, plus the spec to tell ffmpeg to fo 50 -> 25fps.

Basically, I'm looking for the quickest and most 'damage free' way to get
the frame rate conversion but still leave the audio and level of picture
detail OK.

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[iz]

On 30 April 2018 at 13:56, Jim web  wrote:
>
> 1) Use gip to get a 50fps file saved to ram.

You'll want to use the --raw flag with GiP to avoid remuxing to MP4
and tagging, both of which create a file copy, which would halve the
max size of programmes you could download to ram disk.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

I've been discussing the 'loss' of the 1280x720 25fps version with someone
at the BBC.

IIUC this stemmed from 'Red Bee' days of yore, and until recently people at
the BBC had thought they had stopped it long ago. Someone apparently
noticed recently that it was still available. And then actually disabled
it. Hence the mysterious recent ending of its availablity.

I have made the point that the 1280x720 25fps version is useful for people
with a content 'cap' problem, etc, so will be missed because the 50fps
version means somewhat bigger files and/or stream rates. But I doubt this
will cause a rethink.

That leaves me currently in a dither between fetching the 1280x720 50fps
and running it though a frame rare conversion or giving up and making do
with the 960x540 version(s). It has also set me wondering about arranging
for gip to fetch to ram storage and then convert that into a file on my
main disc. (The machine I use does have 8GB of ram, mostly free when I am
fetching things in the morning.) Anyone have experience of doing this, file
by file?

What I have in mind is a process of

1) Use gip to get a 50fps file saved to ram.

2) Get ffmpeg to convert that into a 25fps version on main disc.

3) Delete the file in ram

4) Loop back to (1) for the next file on the list.

I can avoid a 'cap' as I do all the fetching before the 9am start of our
'metered' period. The above should reduce wear and tear on my main disc (an
SSD). But I don't know how feasible it might be. Comments?...

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[chris chery]

hi i m in france
I just try to download hamlet  again to see if i could get the hlshd1 format 
,which I did get  but all other download attempts gave me hvfxsd1
- Original Message - 
From: Paul Thornett

To: chris chery
Sent: Wednesday, April 18, 2018 2:44 PM
Subject: Re: no more hslv format ?


Just because I'm interested, where are you located (I'm in Australia)? And 
how come you need to download a 2nd time?


I downloaded Hamlet in hlshd format. It's 3.2Gb, and I'm happy to share it 
with you if you like. I could also reduce its size using any parameters you 
like.



-
Regards,
 Paul Thornett



On Wed., 18 Apr. 2018, 21:55 chris chery, <chrisch...@free.fr> wrote:

hi
whatever happened on or arounf friday 13 th april when I downloaded  hamlet
in hlsv 1200*720 25fpsand now when all I get is  the Hvfxsd1format 960*540
25dfps
Trying to redownload hamlet  i do get the hlsv size but no such luck with
any other progammes
i am outside the uk
regards
cc
- Original Message - 
From: "Paul Thornett" <pthorn...@gmail.com>

To: "RS" <richard...@zoho.com>
Cc: <get_iplayer@lists.infradead.org>
Sent: Wednesday, April 18, 2018 1:26 PM
Subject: Re: Cannot play downloads from get_iplayer!



That's interesting and odd. With the --info option I don't see the
hlsxx options nor the hvf... one, but I can certainly download the
hlsxx one!
Regards,

Paul Thornett


On 18 April 2018 at 20:41, RS <richard...@zoho.com> wrote:

On 18/04/18 11:05, Paul Thornett wrote:


As a matter of interest, how do I check if hls is available for a
particular file? I thought --info would show this, but using this
option on "Ordeal by Innocence Episode 1" does not reveal the
availability of the hlshd stream, even though it is there.



This is what --info gives me for that episode.
modes:  editorial:
dvfhd1,dvfhd2,dvfsd1,dvfsd2,dvfxsd1,dvfxsd2,dvfxhigh1,dvfxhigh2,
dvflow1,dvflow2,hlshd1,hlsvhigh1,hvfhd1,hvfhd2,hvfhd3,hvfsd1,hvfsd2,
hvfsd3,hvfxsd1,hvfxsd2,hvfxsd3,hvfhigh1,hvfhigh2,hvfhigh3,hvfxhigh1,
hvfxhigh2,hvfxhigh3,hvfstd1,hvfstd2,hvfstd3,hvflow1,hvflow2,hvflow3,
subtitles1,subtitles2,subtitles3

[line breaks inserted]

I have just noticed that the only HLS modes are hlshd1 and hlsvhigh1. For
the other two episodes there were only DVF and HVF modes.


Best wishes
Richard


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer 



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer