Re: no more hslv format ?

[RS]

On 03/05/18 12:18, Steve Dodd wrote:


Is it possible it depends on the source material? From the BBC quote
earlier it sounds like their "source" material is still mostly HD
interlaced, but perhaps some of their sources are all also
progressive, and those ones get doubled identical frames (which might
be a logical result of feeding 25p material into something that's
designed to interpolate interlaced fields into frames)?

The satellite broadcasts I have seen from the BBC and other broadcasters 
are 1920x1080i25 for HD and 720x576i25 or 704x576i25 for SD.  Everything 
that is broadcast is in an interlaced format at some stage, however it 
is generated.  Some programmes are only produced for the iPlayer, so 
they may be generated in a different way.


Best wishes
Richard



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[RS]

On 03/05/18 11:41, Ralph Corderoy wrote:


ffmpeg can produce PNGs, one per frame, and convert only a few specific
seconds to avoid tens of thousands of them, and then adjacent frames
could be compared in pairs to see if they are indeed identical;  compare
`ab', `bc', `cd'...

There's a couple of variants for comparing images.

 compare foo.png bar.png x:
 gm compare -highlight-style assign foo.png bar.png -file x:


Hi Ralph

Would you first need to convert the H.264 or H.265 to raw video?  If one 
PNG is of an I-frame and the next is a P-frame or B-frame they are bound 
to be different.


Best wishes
Richard


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[iz]

On 2 May 2018 at 16:49, Jim web  wrote:
>
> As things are, I could then have gip+ffmpeg generate the 50 fps mp4 files
> (i.e. in the current form) onto hd. This saves some hd wear but means I now
> have those (big) files on hd before I then run a 50 -> 25 fps process to
> create smaller ones (which will be akin in size to what I got from the
> 25fps fetching). The result using this approach means I've still done the
> fetching as quickly as possible, but have about three times as much data
> written to hd at some point.

You may be able to limit the whole process to only two file writes
with a little scripting and XML parsing. Use --raw with GiP  to save
the .ts file directly to drive or to drive via ram disk. The ram disk
probably wouldn't be of much use since I doubt your downloads are
limited by disk write speed. Also use --metadata to save programme
metadata in an XML file and --thumbnail to save the cover art in a JPG
file. Write a script to re-encode the file to 25fps MP4 and at the
same time add metadata tags with ffmpeg, using the XML file and JPG
file as input to construction of the ffmpeg command string. You'll
need ffmpeg 4.0 or higher to add cover art to MP4.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Steve Dodd]

On Thu, May 3, 2018 at 12:24 AM, RS  wrote:

> I think it was Nick Payne who said he had experimented with re-encoding in 
> HEVC (H.265) and found that the file size was the same for 25fps as it was 
> for 50fps, which led him to conclude that frames were being duplicated to 
> achieve 50fps.

Is it possible it depends on the source material? From the BBC quote
earlier it sounds like their "source" material is still mostly HD
interlaced, but perhaps some of their sources are all also
progressive, and those ones get doubled identical frames (which might
be a logical result of feeding 25p material into something that's
designed to interpolate interlaced fields into frames)?

S.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Richard,

> I think it was Nick Payne who said he had experimented with
> re-encoding in HEVC (H.265) and found that the file size was the same
> for 25fps as it was for 50fps, which led him to conclude that frames
> were being duplicated to achieve 50fps.

ffmpeg can produce PNGs, one per frame, and convert only a few specific
seconds to avoid tens of thousands of them, and then adjacent frames
could be compared in pairs to see if they are indeed identical;  compare
`ab', `bc', `cd'...

There's a couple of variants for comparing images.

compare foo.png bar.png x:
gm compare -highlight-style assign foo.png bar.png -file x:

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Ralph Corderoy]

Hi Nick,

> I had some of the coverage of the UK snooker championships
...
> 4h58m match: D/L size 5.03Gb @ 25fps, output from Handbrake was 1.55Gb
> 4h44m match: D/L size 10.2Gb @ 50fps, output from Handbrake was 1.48Gb

Perhaps many frames(!) of snooker coverage isn't ideal for this
comparison as it often is a static picture of the table.  :-)

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article <20180502152015.50ea621...@orac.inputplus.co.uk>, Ralph
Corderoy  wrote:
> > The challenge for me is to work out how to get the fetched file to go
> > onto the tmpfs

> Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> `--output /tmp' and you should see its intermediate files, and the final
> file, only appear there.  `--command' could then move that final file to
> the SSD, or run a conversion command that writes to the SSD and then
> removes the tmpfs input.

the df command listed various 'locations'. These included

tmpfs  815696  1352  814334  /run

none 4078460  0  4078460  /run/shm

none 102400  8  102392  /run/usr

Inside /run/usr I did find a directory for my user id and could use that as
the user. However I don't know what the /run/shm is for, so need to do some
finding out...

I did set up an explict ramdisc on another machine ages ago by adding a
line to the /etc/fstab file. But that fixed the size to 256 MB, which in
this context is tiny. System monitor says I have 8GB of ram.

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: no more hslv format ?

[Jim web]

In article <20180502152015.50ea621...@orac.inputplus.co.uk>, Ralph
Corderoy  wrote:
> >
> > The challenge for me is to work out how to get the fetched file to go
> > onto the tmpfs

> Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> `--output /tmp' and you should see its intermediate files, and the final
> file, only appear there.  `--command' could then move that final file to
> the SSD, or run a conversion command that writes to the SSD and then
> removes the tmpfs input.

Thanks. :-)

> As for altering the ffmpeg command that get_iplayer is using, I'm not
> sure that's worthwhile?  It isn't doing any transcoding, just changing
> the container format, or splicing in better audio, that kind of thing.
> So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second
> lossy one that you'd prefer to combine with the first.  I could be
> wrong, not knowing how to have ffmpeg do this conversion.  When you find
> out, let the list know.  :-)

My concern is that I only have a total of 8GB of ram at present on the
machine, and the 1280x720 50fps files tend to come in at 2GB or more per
hour. 

To save time and avoid running past 9am (and entering metered time) it
would be quickest to do all the fetches first. But this will mean more
files that I can store on ram on some days. So the wish would be to then
shift or process material and put it on hd.

As things are, I could then have gip+ffmpeg generate the 50 fps mp4 files
(i.e. in the current form) onto hd. This saves some hd wear but means I now
have those (big) files on hd before I then run a 50 -> 25 fps process to
create smaller ones (which will be akin in size to what I got from the
25fps fetching). The result using this approach means I've still done the
fetching as quickly as possible, but have about three times as much data
written to hd at some point.

It would be nice to do the process file by file and in each case only the
'final' 25 fps version gets written to hd. This means the same hd use as
previously. But that either means I need enough ram to hold all the 'temp'
files (masses of ram needed) or doing the process item by item. (slow)

I could use spinning rust I guess. but that seems messy. Otherwise it seems
to want me to buy and fit a lot of ram. :-)

So at present I'm wondering and experimenting. 8-]

Jim

-- 
Electronics  https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

All OK

[CJB]

Just refreshed radio and tv caches using the latest GiP. I thought
that May 1 was some kind of cut-off date? Chris B.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer