Re: Audio/Video Out of Sync

[RS]

There seems to have been a new development in the direction of getting 
worse.  BBC Business Live (b07p4t1f) and News at One (b07myk09) HLShd1 have 
both produced --raw .ts files which I can't play with VLC 2.2.4.  With the 
earlier problem files Tools did not show Media or Codec information and it 
did not show the duration, but now I just get a blank screen and no sound. 
I can play them with a program called Movies & TV, which must have come with 
Windows 10 since I haven't installed it, but the video is very jerky after 
the first error.  There are missing segments in News at One at 11, 38, 39, 
43, 57, 58, 59, 64, 65,92, 94, 98, 99, 100, 104, 105, 106, so that has got 
worse again. 




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Vangelis forthnet]

On Thu Aug 18 20:06:01 BST 2016, RS wrote: 

It was using the same server, 
cp401489-vh.akamaihd.net


"cp401489-vh.akamaihd.net" is just a CDN's server hostname; 
it doesn't point to a unique physical server, only.
You can read more about CDNs and Akamai at: 


https://en.wikipedia.org/wiki/Content_delivery_network

https://en.wikipedia.org/wiki/Akamai_Technologies

So, depending on client's actual location, 
different clients may be served by different nodes 
(PoP) of the CDN - at least this is how I 
interpret it, not being a networks expert, of course ;-(


Regards, 
Vangelis.


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: RS Sent: Thursday, August 18, 2016 17:46


That's interesting.  I have tried it again and I get the same missing 
segments as before.  This time I let it continue to the end and the only 
missing segment after 43 was 155.  We do agree on some of them, so I don't 
know what it means.


I did also try it on 3 mobile broadband.  It was painfully slow, 900kbit/s 
for a claimed 4G connection, so I didn't let it run to the end.  The first 6 
errors were the same.  It was using the same server, 
cp401489-vh.akamaihd.net


News at Six was no better.



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Vangelis forthnet
Sent: Thursday, August 18, 2016 15:38


hlshd1 is now present but there are errors at
4, 9, 10, 15, 16, 17, 33, 34, 36, 43 ...




I just tried:



perl get_iplayer-295.pl --pid=b07p4swg --tvmode=hlshd1 --raw



My download had eight (8) missing segments
(for a ca. 25min programme), precisely the following:



10, 15, 16, 43, 93, 152, 153, 155.



So not exactly the same as yours (???)


That's interesting.  I have tried it again and I get the same missing 
segments as before.  This time I let it continue to the end and the only 
missing segment after 43 was 155.  We do agree on some of them, so I don't 
know what it means.


As for the effect, although the subject says out of sync, if you play 
the --raw output (.ts) in VLC, it stops when it gets to a missing segment. 
You can jump over the missing segment and play the next good bit.


Usually I stop when I see there are errors and just throw away the defective 
file.  I then download again in FlashHD.


There is clearly work going on.  As we noted yesterday, the errors have got 
worse.  This morning for BBC Business Live the HLSHD mode was delayed.  For 
the News at One (b07myjtb) both the FlashHD and HLSHD modes were delayed 
about 50 minutes behind the other modes.


The errors I saw for hlshd1 were at 3, 12, 27, 30, 37, 38, 40, 41, 42, 49 
...




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Thursday, August 18, 2016 17:05



The point that was made to me is that the actual audio is HE but is
*misdescribed* in the headers because the header can't correctly describe
this. So it relies on the renderer/client to spot what the data format
really is.


When I looked into AAC-LC and HE-AAC, which is going back a few months now, 
and I thought it was you who explained it to me, my understanding was that 
if a player did not understand the HE-AAC add-ons it could safely ignore 
them.  I thought also it was only used at bit rates below 64kbit/s.


I seem to remember I found this article helpful.
https://en.wikipedia.org/wiki/High-Efficiency_Advanced_Audio_Coding





___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article , Vangelis
forthnet
 wrote:
> On Thu Aug 18 10:06:01 BST 2016, Jim web wrote: 

> > I've been told that the root cause is that these use HE-AAC which the
> > MPEG2 TS spec can't correctly describe. So it is called aac-lc and it
> > is left to the client program to recognise the actual codec.

FWIW I was told the above by someone at the BBC who I'd expect to know.
However they may be wrong. He and I were referring specifically to the
1280x720 50fps streams obtained by hvfhd.

The point that was made to me is that the actual audio is HE but is
*misdescribed* in the headers because the header can't correctly describe
this. So it relies on the renderer/client to spot what the data format
really is. 

>[snip]

> The 50fps hvf modes, 

> hvfsd, hvfhd

> (which are derived from "iptv-all" mediaset) have their audio stream
> encoded as AAC LC @128kbpsABR

> so Spectral Band Replication IS NOT USED on those (that is why the
> encode has increased "transparency").

I also do see it reported as LC when I use ffprobe. But I have been told
the above by someone at the BBC. They may be wrong. It seems an odd thing
for him to say if it is wrong, though. I may also have misunderstood what
he wrote, though, so I'll see if I can check with him.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Vangelis forthnet]

On Thu Aug 18 10:06:01 BST 2016, Jim web wrote: 

I've been told that the root cause is that these use HE-AAC 
which the MPEG2 TS spec can't correctly describe. 
So it is called aac-lc and it is left to the client program 
to recognise the actual codec.


On Thu Aug 18 15:15:23 BST 2016, Jim web wrote: 

However the new ffprobe reports the result 
as having an aac lc soundstream. Not he. 
So it seems to have left something set as being aac lc
despite the actual sound being aac he. 
Am I misunderstanding something here? 


... The audio stream of ALL flash and hls tvmodes is 
AAC LC @96kbpsABR. 

It's somewhat different with hvf tvmodes: 

The lower resolution modes, 

hvflow, hvfstd, hvfhigh, hvfvhigh 

(which are derived from the 
"apple-ipad-hls" mediaset) 
have their audio stream encoded as 
HE-AACv1 @96kbpsABR


High Efficiency (HE) is just an 
encoding profile of AAC LC aimed 
at lower bitrates (it is also called 
SBR, aac+ etc., see: 


https://en.wikipedia.org/wiki/High-Efficiency_Advanced_Audio_Coding

It is an extension of Low Complexity AAC (AAC LC) 
optimized for low-bitrate applications such as streaming audio


HE-AAC decoder is very 
popular among mobile devices...


The 50fps hvf modes, 


hvfsd, hvfhd

(which are derived from "iptv-all" mediaset) 
have their audio stream encoded as 
AAC LC @128kbpsABR


so Spectral Band Replication IS NOT USED 
on those (that is why the encode has increased 
"transparency").


I do not have a flavour of Linux to test, 
but from what I've read in the GiP Support 
Forum, the loss of audio (with some players) 
in the "avconv" remuxed MP4 files
is just due to avconv's inability to properly 
remux the type of streams contained in the 
beeb's hvf modes (which have DTS & PTS 
disparities...).


Regards, 
Vangelis.


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Vangelis forthnet]

On Thu Aug 18 11:39:35 BST 2016, RS wrote:


hlshd1 is now present but there are errors at
4, 9, 10, 15, 16, 17, 33, 34, 36, 43 ...


... Just out of curiosity (and since I'm not a cat),
I just tried:

perl get_iplayer-295.pl --pid=b07p4swg --tvmode=hlshd1 --raw

My download had eight (8) missing segments
(for a ca. 25min programme), precisely the following:

10, 15, 16, 43, 93, 152, 153, 155.

So not exactly the same as yours (???)

I then used ffmpeg (v3.0) to demux (extract)
elementary streams from the MPEG-TS container.

Elementary Video Stream (EVS):

ffmpeg -loglevel 8 -stats -i Video.ts -an -c:v copy -f h264 EVS.264

ffmpeg reported the duration of the video stream to be 25min21sec:

frame=37032 fps=1013 q=-1.0 Lsize=  409012kB time=00:25:21.24 
bitrate=2202.6kbits/s


Elementary Audio Stream (EAS):

ffmpeg -loglevel 8 -stats -i Video.ts -vn -c:a copy -f adts EAS.aac

ffmpeg reported the duration of the audio stream to be 24min41sec:

size=   17374kB time=00:24:41.28 bitrate=  96.1kbits/s

while the duration of the downloaded "RAW" Video.ts file
is reported by MediaInfo (and MPC-HC) to be 00:26:01.

So, as reported elsewhere, it is those inconsistencies between
reported durations of the initial container (I am not sure but
maybe the container duration is set by server data?) and the actual
durations of the truncated (due to missing fragments) elementary
streams that is causing the loss of video/audio sync in the
"one step" remux (change of container format from TS => MP4)
that is occuring in GiP by default...

If I then remux (repackage) the elementary streams to
the MP4 container:

ffmpeg -loglevel 8 -stats -i EVS.264 -i EAS.aac -c copy -bsf:a aac_adtstoasc 
ff-remux.mp4


the end .mp4 file has no loss of sync,
MediaInfo and MPC-HC report the file's
duration to be 00:24:41 (the same as EAS.aac).

Hope the above is useful to someone...

Vangelis. 



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <55b1856f52...@audiomisc.co.uk>, Jim web

wrote:

> > Example... get_iplayer --mode=hlsstd --type=radio --verbose --force
> > --ffmpeg /home/user/ffmpeg http://www.bbc.co.uk/programmes/b07nn8kd

> Excellent! :-)  Thanks very much. I've now built the version of ffmpeg I
> obtained via git this morning. All being well, I'll give this a try
> later today.

I've just tried this and... t'werks! 8-)

I put the current version of ffmpeg at my usual location in userland and
used the --fmpeg option to point to it when doing an hvfhd fetch.

The resulting mp4 file plays nicely with VLC. That *might* be a fluke as
one previous other example did. But it is promising.

However the new ffprobe reports the result as having an aac lc sound
stream. Not he. So it seems to have left something set as being aac lc
despite the actual sound being aac he. Am I misunderstanding something
here?

The new ffprobe when given a ts obtained with -raw tells me the header is
malformed wrt the sound but then gives a lot of errors without telling me
the details. I guess I should tell it to use a longer analysis limit?

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[David Cantrell]

On Tue, Aug 16, 2016 at 09:44:43PM +0200, iz wrote:
> > >From: David Cantrell
> > >> >I'd like to rate-limit get_iplayer so that other things on the same
> > >> >machine that are also talking to the internet don't run like stunned 
> If you are using OS X, this approach worked for me:
> https://dreness.com/blog/archives/843
> 
> Just change "dummynet out" to "dummynet in" and change the bandwidth limit to 
> desired value.

Excellent, thanks! That looks like a very nice solution.

-- 
David Cantrell | top google result for "topless karaoke murders"

If you have received this email in error, please add some nutmeg
and egg whites, whisk, and place in a warm oven for 40 minutes.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[David Cantrell]

On Tue, Aug 16, 2016 at 10:03:20PM +0100, RS wrote:

> Maybe I'm missing something, but I find that incredible.  The Mac is sold 
> as a machine which allows creative people to create without having to 
> bother with technicalities.

And a Ferrari is sold as something that will turn you into a racing god
and attract members of the appropriate sex, even though it actually does
neither. Please don't confuse marketing with reality :-)

As far as I'm concerned a Mac is a good compromise between Unix, doing
A/V playback reliably, and having commercial desktop software available.

> If I search for Linux traffic management or 
> Windows traffic management I can see that there are programs which do that 
> sort of thing, but they are only needed if I want to do clever things like 
> running a heavily loaded server where the load has to be balanced between 
> processes. I have 5 programs open on my task bar and 58 processes.  Windows 
> allocates resources between them as they demand them

OS X does that too. It's just that in this rare case I know better than
the OS does. Exactly the same problem would arise if I used Linux or
FreeBSD or Solaris instead. It's the problem that the --bwlimit option
on rsync exists to solve.

-- 
David Cantrell | Enforcer, South London Linguistic Massive

If you have received this email in error, please add some nutmeg
and egg whites, whisk, and place in a warm oven for 40 minutes.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article , RS
 wrote:
> >From: Jim web Sent: Thursday, August 18, 2016 09:44

> >But the actual application of any fix will be being done by someone
> >else. People at the BBC can, of course, check the output results. But
> >may not be able to check the details of what is being actually done by
> >others.

> BBC Business Live 0830-0900 BBC News (b07p4swg) took longer to appear
> today than yesterday.  Its best HLS is only  HLSvhigh and still has
> lots of errors at  3, 7, 8, 16, 17, 22, 23, 25,  26, 27, 28, 32, 35 ...

OK, thanks. I've not heard back from my BBC contact on this. But I there
would probably be delays as he passes my questions back along the chain,
and then gets a reaction back once the people at the other end respond. My
BBC contact seemed confident the fix was going into place, and would work.
But this may have been what he'd been told and had no reason at the time to
doubt it!

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <1569d0d6842.c393cad6181596.4714555984937837...@zoho.com>,
   batguano999  wrote:
> Hi If you have a version of FFmpeg, either compiled and installed by
> yourself or a static one in a folder somewhere downloaded from
> http://johnvansickle.com/ffmpeg/ You can tell get_iplayer to use it each
> time (or set it for good in the get_iplayer preferences). External
> Program Options: --ffmpeg   Location of ffmpeg or
> avconv binary. Synonyms: --avconv

> Example... get_iplayer --mode=hlsstd --type=radio --verbose --force
> --ffmpeg /home/user/ffmpeg http://www.bbc.co.uk/programmes/b07nn8kd

Excellent! :-)  Thanks very much. I've now built the version of ffmpeg I
obtained via git this morning. All being well, I'll give this a try later
today.

Thanks,

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: RS Sent: Thursday, August 18, 2016 10:52


BBC Business Live 0830-0900 BBC News (b07p4swg) took longer to appear today 
than yesterday.  Its best HLS is only  HLSvhigh and still has lots of 
errors at  3, 7, 8, 16, 17, 22, 23, 25,  26, 27, 28, 32, 35 ...


hlshd1 is now present but there are errors at 4, 9, 10, 15, 16, 17, 33, 34, 
36, 43 ...






___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Thursday, August 18, 2016 09:44



But the actual
application of any fix will be being done by someone else. People at the
BBC can, of course, check the output results. But may not be able to check
the details of what is being actually done by others.


BBC Business Live 0830-0900 BBC News (b07p4swg) took longer to appear today 
than yesterday.  Its best HLS is only  HLSvhigh and still has lots of errors 
at  3, 7, 8, 16, 17, 22, 23, 25,  26, 27, 28, 32, 35 ...


FlashHD1 still seems alright.





___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Re: Audio/Video Out of Sync

[batguano999]

> 
>I'll git and build a current version of ffmpeg as it is about time I did 
>this anyway. But the gip related question this raises in my mind is that, 
>having done this, how best to have gip use such a new version for its 
>after-fetch conversions? 
>

Hi
If you have a version of FFmpeg, either compiled and installed by yourself or a 
static one in a folder somewhere downloaded from 
http://johnvansickle.com/ffmpeg/
You can tell get_iplayer to use it each time (or set it for good in the 
get_iplayer preferences).
External Program Options:
 --ffmpeg   Location of ffmpeg or avconv binary. 
Synonyms: --avconv

Example...
get_iplayer --mode=hlsstd --type=radio --verbose --force --ffmpeg 
/home/user/ffmpeg http://www.bbc.co.uk/programmes/b07nn8kd

See from the paste ---> http://pastebin.com/raw/JUY8ZeWg

It has used the static FFmpeg...
ffmpeg version N-81378-gbba6a03-static http://johnvansickle.com/ffmpeg/  
Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.1 (Debian 5.4.1-1) 20160803

Instead of my own (installed) FFmpeg...
ffmpeg version git-2016-08-17-fabff10 Copyright (c) 2000-2016 the FFmpeg 
developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
,
   tellyaddict  wrote:
> Thanks Jim,

> > Afraid the info I have is limited. But it makes clear that the BBC
> > realise there *is* a problem with the hls segmentation and that Akamai
> > are said to have a fix which is supposed to be being put in place.
> > However I'd have thought that would happen today. Might be my
> > misunderstanding, though.

> As this problem appears to have got significantly worse today, I think
> it's possible that something was attempted but hasn't worked.

Yes, that may be the case. I've not heard more as yet...

> > A problem here is that I'm just one stage of a 'pass the parcel' when
> > it comes to this. IIUC The fix isn't one the BBC themselves are doing,
> > but someone else who is responding to them.

> From what you have said before, I get the feeling that the BBC have no
> control over this problem because on that legacy system, all they (the
> BBC) do is send the video files to Akamai. Akamai does the rest
> (segmenting, uploading etc) so the problems start when Akamai start
> processing. So Akamai are the only ones who can actually fix this
> problem. 

Yes. That is my understanding.

> As we have seen from HVF and DASH, these are segmented,
> packaged etc by the BBC and then uploaded to Akamai's servers, and it
> seems that things processed by the BBC tend to work.

Yes. Again, that is what I think is the case.

My impressions are that I think it was useful for the BBC that we contacted
them to raise this. It probably helped them to realise the problems were
ocurring. They have then interacted with Akamai, I assume. But the actual
application of any fix will be being done by someone else. People at the
BBC can, of course, check the output results. But may not be able to check
the details of what is being actually done by others.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

I now have some info that helps me understand the 'missing audio' problem
with the hvfhd fetched files.

I've been told that the root cause is that these use HE-AAC which the MPEG2
TS spec can't correctly describe. So it is called aac-lc and it is left to
the client program to recognise the actual codec.

This explains why ffprobe can find it as it checks the actual streams. But
apparently my copy of vlc (which uses the distro installed avconv) doesn't,
and then can't make sense of the audio.

I'll git and build a current version of ffmpeg as it is about time I did
this anyway. But the gip related question this raises in my mind is that,
having done this, how best to have gip use such a new version for its
after-fetch conversions? I'm hoping that will then allow it to generate mp4
files from the ts fetched which correctly describe the audio format.

 I'll have a look at the webpages, etc, but don't know if they deal with
this. So maybe someone here can say?

FWIW Usually I leave the distro installed versions of such things in place
and just keep a copy of the 'new' ffmpeg/ffprobe/ffplay in a user directory
and call them from there. This avoids upgrading in a way that breaks
something else, and means I can easily compare different versions.

That said, the xfce Mint distro on my 'AV' machine seems to have avconv
installed, but not the ffmpeg family. Need to check that, though.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

RE: Audio/Video Out of Sync

[George Eycott]

> base on an unsupported hypothesis that get_iplayer only fetches invalid
> chunks because it is fetching much faster than any real client.

My "unsupported" hypothesis was based on the fact that normal IPlayer users
do not seem to be having (or rather are not noticing) the problem and the
speed of file retrieval was an obvious difference. It may be wrong (and as
further information has come to light, it looks like it probably was wrong),
but at the time I (and others) thought it was worth investigating.

> I offer as a counter example the iPlayer built into my Humax HDR Fox T2
PVR.
> This fetches the programme absolutely as fast as it can, totally saturates
my
> broadband link doing it, and saves it to file. It plays the programme from
the
> file, and provided the fetch proceeds faster than the playback it doesn't
stall.
> But it does nothing to stop the fetch getting way ahead of playback, which
it
> does on my 79mbps downstream FTTC.

But is getting missing segments? Depending on how it treats the file it may
or may not end up with the sync problems so you may not notice the problem.
Of course it may or may not be using one of the problem formats in the first
place. Sadly my broadband (well they call it broadband) is so slow that just
sending this email will saturate it for a noticeable period, I have not
bothered connecting my Humax to the internet, life is too short!

Anyway, thanks to contacts within the list we now know more (or at least we
now know it is an issue that is being looked at) than we did then so indeed,
my hypothesis was incorrect, doesn't mean it wasn't worth testing at the
time though.

Cheers

George




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

Thanks Jim,

> Afraid the info I have is limited. But it makes clear that the BBC realise
> there *is* a problem with the hls segmentation and that Akamai are said to
> have a fix which is supposed to be being put in place. However I'd have
> thought that would happen today. Might be my misunderstanding, though.

As this problem appears to have got significantly worse today, I think it's 
possible that something was attempted but hasn't worked.

> A problem here is that I'm just one stage of a 'pass the parcel' when it
> comes to this. IIUC The fix isn't one the BBC themselves are doing, but
> someone else who is responding to them.

>From what you have said before, I get the feeling that the BBC have no control 
>over this problem because on that legacy system, all they (the BBC) do is send 
>the video files to Akamai. Akamai does the rest (segmenting, uploading etc) so 
>the problems start when Akamai start processing. So Akamai are the only ones 
>who can actually fix this problem. As we have seen from HVF and DASH, these 
>are segmented, packaged etc by the BBC and then uploaded to Akamai's servers, 
>and it seems that things processed by the BBC tend to work.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

RE: Audio/Video Out of Sync

[MacFH - C E Macfarlane]

Please see below ...

> -Original Message-
> From: get_iplayer [mailto:get_iplayer-boun...@lists.infradead.org] On
> Behalf Of Vangelis forthnet
> Sent: 17 August 2016 09:58
> To: get_iplayer@lists.infradead.org
> Subject: Re: Audio/Video Out of Sync
> 
> On Tue Aug 16 20:25:00 BST 2016, C E Macfarlane wrote:
> 
> > I was searching each night's log for the word 'error', but in fact the
> > missing segment messages come up as 'WARNING'.
> > It's not a big problem now that I've realised my mistake, but I think
> > that message ought to be changed to 'ERROR', because it means that the
> > resulting file will be corrupted.
> 
> Hello Charles, I hope all's well with you!

Yes, thanks, apart from my old laptop dying  -  it's a bummer, but I got
more life and use from it than I could ever have dared to expect for the
used price that I originally paid for it, so I can't really complain!  Hope
all's well with you as well.

> What you suggest has been already implemented in the develop branch
> (2.96-dev) of GiP, specifically with commit git-1564276:
> 
> https://github.com/get-iplayer/get_iplayer/commit/1564276

Excellent!  I'm glad to hear that.  Thanks.



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
,
   tellyaddict  wrote:
> Great so now the programmes are mostly made up of missing segments.
> Hopefully this shows they are trying to do something to the servers to
> fix this issue. Whatever they have done so far though appears to have
> had the opposite effect.

> @jim Do you have any info on what the problem is or what's being done
> about it?

Afraid the info I have is limited. But it makes clear that the BBC realise
there *is* a problem with the hls segmentation and that Akamai are said to
have a fix which is supposed to be being put in place. However I'd have
thought that would happen today. Might be my misunderstanding, though.

What I don't know is if they haven't actually carried this out yet, or if
the problems need flushing though in some way. I've reported that people
still have the problem. With luck I'll hear more tomorrow... or the fix
will be in place and work.

A problem here is that I'm just one stage of a 'pass the parcel' when it
comes to this. IIUC The fix isn't one the BBC themselves are doing, but
someone else who is responding to them.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <04EBC8DD8B6A48BCA3403FECE9D1FC59@RJCDESK>, RS
 wrote:


> If anything it seems to be getting much worse.  I tried the BBC1
> national News at One (b07myjll).  There were errors at 2, 3, 6, 25, 33,
> 39, 43, 44, 63, 71, 72, 73 ...

> The errors are getting closer together.

OK, I've asked someone about this. With luck I'll get a response tomorrow
morning.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

Great so now the programmes are mostly made up of missing segments. Hopefully 
this shows they are trying to do something to the servers to fix this issue. 
Whatever they have done so far though appears to have had the opposite effect.

@jim Do you have any info on what the problem is or what's being done about it?

___

> >> I have just tried BBC Business Live broadcast at 0830 to 0900 on BBC
> >> News (b07p4sqp) this morning again hlshd1.  It had errors at 9, 10, 12,
> >> 13, 21, 22, 35, 39, 44, 48 ...
> 
> >OK. Thanks. Whatever the reasons, the problems seem to be continuing at
> >present.
> 
> If anything it seems to be getting much worse.  I tried the BBC1 national 
> News at One (b07myjll).  There were errors at 2, 3, 6, 25, 33, 39, 43, 44, 
> 63, 71, 72, 73 ...
> 
> The errors are getting closer together.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Wednesday, August 17, 2016 16:05



I have just tried BBC Business Live broadcast at 0830 to 0900 on BBC
News (b07p4sqp) this morning again hlshd1.  It had errors at 9, 10, 12,
13, 21, 22, 35, 39, 44, 48 ...



OK. Thanks. Whatever the reasons, the problems seem to be continuing at
present.


If anything it seems to be getting much worse.  I tried the BBC1 national 
News at One (b07myjll).  There were errors at 2, 3, 6, 25, 33, 39, 43, 44, 
63, 71, 72, 73 ...


The errors are getting closer together.



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article , RS
 wrote:
> >From: Jim web Sent: Wednesday, August 17, 2016 13:17

> >I don't know from the pid when b07pmyd6 was broadcast or put on-server.
> >Was that last night? If so, it looks like the fix hasn't been done yet,
> >or hasn't worked. I'll ask someone about it... but note that they in
> >turn have to refer though others.

> b07pmyd6 was Inside the Factory broadcast from 2000 to 2100 on BBC2 last
> night.  It is a new programme but could still have been made months ago.

I can't be certain at this point because my focus has tended to be on the
'Audio Factory' rather than the video. But by default I'd assume material
is injected into the iplayer system from the 'live feeds'. i.e. what is
being sent to TX.

> I have just tried BBC Business Live broadcast at 0830 to 0900 on BBC
> News (b07p4sqp) this morning again hlshd1.  It had errors at 9, 10, 12,
> 13, 21, 22, 35, 39, 44, 48 ...

OK. Thanks. Whatever the reasons, the problems seem to be continuing at
present.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Vangelis forthnet]

On Wed Aug 17 13:17:46 BST 2016, Jim web wrote: 


I don't know from the pid when b07pmyd6 was broadcast


... You can use the following template: 


http://www.bbc.co.uk/programmes//broadcasts

ergo

http://www.bbc.co.uk/programmes/b07pmyd6/broadcasts

=> BBC Two, 16 Aug 2016, 20:00 BST

[or use GiP itself: 

get_iplayer --pid=b07pmyd6 -i | FindStr lastbcast 
=>

lastbcast:  original: 2016-08-16T20:00:00+01:00

(on Linux Mint replace FindStr with grep)]

Cheers

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Wednesday, August 17, 2016 13:17



I don't know from the pid when b07pmyd6 was broadcast or put on-server. Was
that last night? If so, it looks like the fix hasn't been done yet, or
hasn't worked. I'll ask someone about it... but note that they in turn have
to refer though others.


b07pmyd6 was Inside the Factory broadcast from 2000 to 2100 on BBC2 last 
night.  It is a new programme but could still have been made months ago.


I have just tried BBC Business Live broadcast at 0830 to 0900 on BBC News 
(b07p4sqp) this morning again hlshd1.  It had errors at 9, 10, 12, 13, 21, 
22, 35, 39, 44, 48 ...





___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <1CF29C0BD25A48F5A6104EEDFD6C635F@RJCDESK>, RS
 wrote:
> >From: Jim web Sent: Wednesday, August 17, 2016 10:03

> >> Yes, that may be what's happening. I am using a relatively old LTS
> >> version of Mint. I don't want to upgrade distro juts for this, so
> >> I'll see if I can update either VLC or avconv.

> Is this from https://github.com/get-iplayer/get_iplayer/wiki/release295
> a repeat of the same information, or is it a new gloss on it?

> "There is one very important limitation: Downloads from Video Factory
> streams cannot be re-muxed to MP4 by avconv or ffmpeg <2.5. Only ffmpeg
> 2.5+ appears to work. 

I'll need to check what version of avconv/ffmpeg is installed on my
relevant machine. However it has reprocessed all the gip fetches since
install. And - until I started with the 50fps files obtained using hcfhd -
the results all play fine with VLC.

> As for the missing segments error, b0377vl5 still has lots of missing
> segments, 63, 64, 69, 73, 80 ... in HLShd1, but that's quite old. 
> b07pmyd6 in HLShd1 from last night has errors at 124, 129, 152, 184 ...

I don't know from the pid when b07pmyd6 was broadcast or put on-server. Was
that last night? If so, it looks like the fix hasn't been done yet, or
hasn't worked. I'll ask someone about it... but note that they in turn have
to refer though others. 

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Wednesday, August 17, 2016 10:03



Yes, that may be what's happening. I am using a relatively old LTS
version of Mint. I don't want to upgrade distro juts for this, so I'll
see if I can update either VLC or avconv.


Is this from
https://github.com/get-iplayer/get_iplayer/wiki/release295
a repeat of the same information, or is it a new gloss on it?

"There is one very important limitation: Downloads from Video Factory 
streams cannot be re-muxed to MP4 by avconv or ffmpeg <2.5. Only ffmpeg 2.5+ 
appears to work. This is of particular note to users still on old LTS 
versions of Ubuntu (12.04, 14.04) and Mint (13, 17), and Mageia 5. If you 
are unable or unwilling to upgrade your system, you will need to acquire a 
modern version of ffmpeg if you wish create MP4 files from Video Factory 
streams (you can still download with --raw). There may be several 
alternatives available, but one possible solution is to use a static build 
of ffmpeg from:


http://johnvansickle.com/ffmpeg/;

As for the missing segments error, b0377vl5 still has lots of missing 
segments, 63, 64, 69, 73, 80 ... in HLShd1, but that's quite old.  b07pmyd6 
in HLShd1 from last night has errors at 124, 129, 152, 184 ...





___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <55b083fabc...@audiomisc.co.uk>, Jim web

wrote:

> Yes, that may be what's happening. I am using a relatively old LTS
> version of Mint. I don't want to upgrade distro juts for this, so I'll
> see if I can update either VLC or avconv.

Update on this:

This morning I refetched some files using hvfhd but this time using the
--raw option.

The resulting .ts files play fine with my copy of VLC!

So the situation seems to be here that the 'same' programme fetched using
hvfhd will play OK as the 'raw' ts but that allowing the usual conversion
to mp4 causes the audio track to be not be played by VLC.

This does solve my problem with playing the hvfhd results on a practical
level. But remains an oddity.

I spent the time this morning getting ts files this way. So I don't know if
the hls segmentation problem has been fixed. Anyone know?

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Vangelis forthnet]

On Tue Aug 16 20:25:00 BST 2016, C E Macfarlane wrote: 

I was searching each night's log for the word 'error', 
but in fact the missing segment messages come up as 'WARNING'.  
It's not a big problem now that I've realised my mistake, 
but I think that message ought to be changed to 'ERROR', 
because it means that the resulting file will be corrupted.


Hello Charles, I hope all's well with you!

What you suggest has been already implemented 
in the develop branch (2.96-dev) of GiP, 
specifically with commit git-1564276: 


https://github.com/get-iplayer/get_iplayer/commit/1564276

Things in the develop branch are changing constantly currently, 
literally by-the-day! 
If anyone is to try it, FIRST READ ALL THE COMMIT TITLES in 


https://github.com/get-iplayer/get_iplayer/commits/develop/get_iplayer

(after "bump dev version")
and the commit summaries by clicking individual commits/snapshots.

If the hls tvmodes issues persist over time, 
then the next major GiP release will be out 
sooner rather than later!


Kind regards, 
Vangelis


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Owen Smith]

I believe this approach is specifically to slow down get_iplayer's use of the 
network far below what would be considered it's reasonable share. This is based 
on a request someone made to be able to do this, base on an unsupported 
hypothesis that get_iplayer only fetches invalid chunks because it is fetching 
much faster than any real client.

I offer as a counter example the iPlayer built into my Humax HDR Fox T2 PVR. 
This fetches the programme absolutely as fast as it can, totally saturates my 
broadband link doing it, and saves it to file. It plays the programme from the 
file, and provided the fetch proceeds faster than the playback it doesn't 
stall. But it does nothing to stop the fetch getting way ahead of playback, 
which it does on my 79mbps downstream FTTC.

-- 
Owen Smith 
Cambridge, UK

On 16 Aug 2016, at 22:03, RS  wrote:

>> From: iz
>> Sent: Tuesday, August 16, 2016 20:44
> 
>> If you are using OS X, this approach worked for me:
> 
>> https://dreness.com/blog/archives/843
> 
>> Just change "dummynet out" to "dummynet in" and change the bandwidth limit 
>> to desired value.
> 
> Maybe I'm missing something, but I find that incredible.  The Mac is sold as 
> a machine which allows creative people to create without having to bother 
> with technicalities.  If I search for Linux traffic management or Windows 
> traffic management I can see that there are programs which do that sort of 
> thing, but they are only needed if I want to do clever things like running a 
> heavily loaded server where the load has to be balanced between processes. I 
> have 5 programs open on my task bar and 58 processes.  Windows allocates 
> resources between them as they demand them without my even having to think 
> about it, and has been doing a reasonable job of it since Windows 95.  Maybe 
> you and David are doing clever things with your machines, but then I don't 
> understand why you would run get_iplayer on the same machine.
> 
> My inclination would be to buy a Raspberry Pi and run get_iplayer on that.
> 
> 
> 
> 
> ___
> get_iplayer mailing list
> get_iplayer@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: iz
Sent: Tuesday, August 16, 2016 20:44



If you are using OS X, this approach worked for me:



https://dreness.com/blog/archives/843


Just change "dummynet out" to "dummynet in" and change the bandwidth limit 
to desired value.


Maybe I'm missing something, but I find that incredible.  The Mac is sold as 
a machine which allows creative people to create without having to bother 
with technicalities.  If I search for Linux traffic management or Windows 
traffic management I can see that there are programs which do that sort of 
thing, but they are only needed if I want to do clever things like running a 
heavily loaded server where the load has to be balanced between processes. 
I have 5 programs open on my task bar and 58 processes.  Windows allocates 
resources between them as they demand them without my even having to think 
about it, and has been doing a reasonable job of it since Windows 95.  Maybe 
you and David are doing clever things with your machines, but then I don't 
understand why you would run get_iplayer on the same machine.


My inclination would be to buy a Raspberry Pi and run get_iplayer on that.




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[iz]

> Sent: Tuesday, August 16, 2016 at 4:39 PM
> From: RS 
>
> >From: David Cantrell
> >Sent: Tuesday, August 16, 2016 14:42
> 
> >> >I'd like to rate-limit get_iplayer so that other things on the same
> >> >machine that are also talking to the internet don't run like stunned 
> 
> If you are using OS/X I can't comment because I have never used it.  A quick 

If you are using OS X, this approach worked for me:

https://dreness.com/blog/archives/843

Just change "dummynet out" to "dummynet in" and change the bandwidth limit to 
desired value.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

RE: RE: Audio/Video Out of Sync

[MacFH - C E Macfarlane]

Please see below ...

> -Original Message-
> From: get_iplayer [mailto:get_iplayer-boun...@lists.infradead.org] On
> Behalf Of MacFH - C E Macfarlane
> Sent: 14 August 2016 14:19
> To: 'tellyaddict'; 'get_iplayer'
> Subject: RE: RE: Audio/Video Out of Sync
> 
> Just FYI, all overnight downloads have been error-free since that one.

Further just FYI, I've now realised that this was an unwarranted and untrue
claim.  I was searching each night's log for the word 'error', but in fact
the missing segment messages come up as 'WARNING'.  It's not a big problem
now that I've realised my mistake, but I think that message ought to be
changed to 'ERROR', because it means that the resulting file will be
corrupted.

As there are too many other warning messages, I'm now searching for
'segment' instead, and found some messages the day before yesterday in a
Gaelic music programme aired on BBC Alba, so I redownloaded it yesterday,
have just watched it now, and it is corrupt again, and audio/video sync does
indeed get progressively worse after each glitch:
Ceol Ulaidh/Castle Sessions ;
http://www.bbc.co.uk/programmes/b07pd3mt

So now I must try and find out what others have slipped through the net ...
Grrr!

Good to hear from Jim that things might be fixed soon.

As for the claim that it can't be download rates or server loading causing
the glitches because they are still there when one tries to download a
second time, that doesn't follow.  We know that download rates and/or server
loading are not the problem from Jim's previous information from the BBC,
but not for any other reason.  To explain: if server loading was a problem,
it would affect both storing stuff on the servers and downloading off them
equally unless specific, different runtime priorities have been assigned to
each type of activity.  Thus if the server hiccupped while segmenting and
storing a programme, that error in the resulting file(s) would be found by
each attempted download until the corrupted file(s) was/were recreated
somehow or other, while if the server hiccupped while serving a download,
that would essentially be a one-time error that in principle could be fixed
by redownloading and hoping that the server doesn't hiccup again on the
second attempt.  If download rates were the problem, the second type of
error ought to predominate, whereas in fact the first type predominates  -
the errors seem to occur mostly at the stage of creating the file(s) that
are used as the source of subsequent downloads.



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article , Vangelis
forthnet
 wrote:
> On Tue Aug 16 09:28:04 BST 2016, Jim web wrote: 

> >> I think what has caused confusion is that the transition from v2.94
> >> to v2.95 made several changes at the same time.
> > (snip) Yes. My apologies for missing this change.

> Yet, you had been made aware of those changes in a verbose manner with
> my post of  Aug 3rd: 

> http://lists.infradead.org/pipermail/get_iplayer/2016-August/009353.html

[snip]

Yes. Sorry again about missing this. However it is a complex tale in part
because I was using my front end which lets me give it a file with lines
like

t p044abcd
r p934352h
h b0ytytypp

etc.

It then fetches the items, using the chosen set of settings for each, and
puts them into appropriate output directories.

Here I had preset my front end so the first letter chose a preset series of
options. (t for the old 'flash' mode, r for the DASH-MPEG fetching of the
320k radio streams, h for 'hls', etc.)

So for many months I've been using this approach having forgotten the
actual mode settings I specified for each 'letter'! I then installed the
current gip and immediately focused on the hlshd fetching... which I was
doing by issuing commands in a terminal. This threw up the 'no sound'
problems with VLC as well as my looking at the missing segments questions.

 I then switched back to using my front end program to autofetch some items
'as normal' in the morning. Having forgotten I'd used a 'best' setting
rather than something more mode specific for the t mode fetching. And also
having forgotten why this might matter.

Self-made trap. :-/


> > I was then wondering about what 'sequence' of modes would be best
> > for finding the 'best available' version.

> In GiP 2.95 (release), the hls tvmodes are the default; not specifying
> a tvmode or setting --tvmode=best means that GiP will exhaust all hls
> modes (from higher to lower quality) until it finds the best available
> one.

> If you want the best available among the hvf tvmodes, then set
> --tvmode=hvfbest Likewise, use --tvmode=flashbest.

> If hvfhd isn't available (720p50fps), it's a good bet that neither
> hlshd nor flashhd (both 720p25fps) are; I would try myself
> --tvmode=hvfbest to fetch either hvfsd (540p50fps) - if available - or
> hvfvhigh (540p25fps); both flashvhigh/ hlsvhigh (468p25fps), which are
> the next step down from missing flashhd/hlshd, are slightly inferior to
> hvfvhigh...

OK. Thanks. At present I'm waiting to see if the segmentation problems get
fixed. If they do I'll probably use the hls for routine tv but the hvfhd by
preference for things like Proms because of the higher audio bitrate.

> But again, 95% of the answers to your queries here in the list are
> found in the detailed FAQs/ Release Notes that dinkypumpkin
> painstakingly compiled:

> https://github.com/get-iplayer/get_iplayer/wiki/release295

> https://github.com/get-iplayer/get_iplayer/wiki/faq

> DO READ THEM!

I confess I have read them in the past. But tend to find the details
complex and easy to forget.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Vangelis forthnet]

On Tue Aug 16 09:28:04 BST 2016, Jim web wrote: 


I think what has caused confusion is that the transition from v2.94 to
v2.95 made several changes at the same time.

(snip)
Yes. My apologies for missing this change.


Yet, you had been made aware of those changes 
in a verbose manner with my post of  Aug 3rd: 


http://lists.infradead.org/pipermail/get_iplayer/2016-August/009353.html

Excerpts from that:
--

A plethora of further commits/changes were
pushed to the 2.95dev repo since Jan 2016,
all these ended up in final 2.95 and have significantly
changed the behaviour you are currently
experiencing in your "Jan 2.95dev" version.
(snip)
in relation to what concerns you the most:
1. "--type=tv" needs not be specified, as the tv.cache
is searched by default.
2. "--type=radio" can be omitted in 2.95, but in reality shouldn't...
3. HLS (tv|radio) modes are now the DEFAULT -
this change was brought on by
https://github.com/get-iplayer/get_iplayer/commit/673793a
in Feb 2016, so not present in your 295dev snapshot!
(snip)
With 2.95-release, YOU DON'T HAVE TO SPECIFY
--modes=hlshd ; this is now the default.
(NB --tvmode & --modes are valid, --mode is not,
according to longhelp).
In your 2.95dev snapshot,
--modes=best
would fetch the flashhd tvmode (the default then).
To specifically request flashhd in 2.95-release, issue:
--modes=flashhd

--

I was then wondering about 
what 'sequence' of modes would be best 
for finding the 'best available' version.


In GiP 2.95 (release), the hls tvmodes are the default; 
not specifying a tvmode or setting --tvmode=best 
means that GiP will exhaust all hls modes (from higher 
to lower quality) until it finds the best available one.


If you want the best available among the hvf tvmodes, 
then set --tvmode=hvfbest

Likewise, use --tvmode=flashbest.

If hvfhd isn't available (720p50fps), it's a good bet that 
neither hlshd nor flashhd (both 720p25fps) are; I would try 
myself --tvmode=hvfbest to fetch either hvfsd (540p50fps) 
- if available - or hvfvhigh (540p25fps); both flashvhigh/
hlsvhigh (468p25fps), which are the next step down from missing 
flashhd/hlshd, are slightly inferior to hvfvhigh...


But again, 95% of the answers to your queries 
here in the list are found in the detailed FAQs/
Release Notes that dinkypumpkin painstakingly 
compiled:


https://github.com/get-iplayer/get_iplayer/wiki/release295

https://github.com/get-iplayer/get_iplayer/wiki/faq

DO READ THEM!

Best regards, 
Vangelis.


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: David Cantrell
Sent: Tuesday, August 16, 2016 14:42



>I'd like to rate-limit get_iplayer so that other things on the same
>machine that are also talking to the internet don't run like stunned 
>slugs

>>in treacle.
>>Second, I'm not using Linux.
>You don't say which non-Linux operating system you are using.  Your
>description doesn't correspond with any version of Windows, 32 bit or 64
>bit



Yes, it does. This would affect Windows just as much as any other OS.


If you are using OS/X I can't comment because I have never used it.  A quick 
Google search for OS X network problems makes me glad I don't use it.


There are lots of things which may be slowing your network down.  Have you 
moved the WiFi channels away from the defaults to reduce interference?  The 
most likely cause, despite denials, is throttling by your ISP.


I used to be very sceptical about the BBC's claims to have ample bandwidth 
until I moved to Virgin Media.  Earlier in the year I started using the 2.95 
dev version of get_iplayer and was astonished by the speed of its native HLS 
reader.


Even if you are right that get_iplayer is slowing your network, the great 
joy of get_iplayer is that it can be run when your machine and your network 
are not doing anything else.





___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[David Cantrell]

On Tue, Aug 16, 2016 at 12:46:51PM +0100, RS wrote:
> >From: David Cantrell Sent: Monday, August 15, 2016 23:37
> >I'd like to rate-limit get_iplayer so that other things on the same 
> >machine that are also talking to the internet don't run like stunned slugs 
> >in treacle.
> >Second, I'm not using Linux.
> You don't say which non-Linux operating system you are using.  Your 
> description doesn't correspond with any version of Windows, 32 bit or 64 
> bit

Yes, it does. This would affect Windows just as much as any other OS.

> I have used except maybe Windows 3.1.  Here downloading HLSHD with 
> get_iplayer at speeds up to 80Mbit/s on an elderly 2.33GHz Core 2 Duo with 
> 4GByte of RAM in 64 bit Windows 10 Home and a 10/100 LAN card gave a CPU 
> usage of between 20% and 52% and memory usage barely above background.  How 
> much RAM have you got?

CPU and RAM are not the limiting factors. Network bandwidth is. Other
stuff is slowing down because it's waiting to send or receive packets.

-- 
David Cantrell | Cake Smuggler Extraordinaire

fdisk format reinstall, doo-dah, doo-dah;
fdisk format reinstall, it's the Windows way

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

Sounds positive :)

Any ideas what the problems are and why they've taken so long to fix?

> Had an email this morning. Just to say that, with some luck, we may find
> that the missing segments problem could be cured soon. Fingers crossed...

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

> I've never tried the --raw option. Might it help with the following?
> 
> A number of the files I fetched yesterday morning had reported missing
> segments. This specified time offsets in secs. I had a look at those
> points.
> 
> In some cases there was no sign of a problem. In others the video froze
> for, say, ten sec, and when it resumed the video and audio seemed still in
> synch. But in other cases the synch was out.
> 
> Would the synch offset effect be avoided by using --raw?

It should help yes :)

The idea is that when FFMpeg repackages the programme into a different format, 
it gets confused by the odd timestamps on the video and audio when there is a 
missing segment. Things then start to drift.

> If so, what other problems might that give? My first thought might be that
> the result would play with something like VLC without any timing. i.e. the
> total and elapsed times would be zero and attempts to jump to some point
> could cause problems with replay.

VLC seems to be coping fine with them. You can move through the file as normal.

> My assumption is that the pass though ffmpeg at the end of a fetch is to
> tidy up these issues. (It is something I have to do with ts files grabbed
> from DVB-T2.) But I am assuming/guessing this.

I think it's also to make files more compatible with different devices. Most 
probably wouldn't know what to do with a .ts file.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <42FEE022394C42678F695912B6A2FFEC@RJCDESK>, RS
 wrote:
> >From: Jim web Sent: Monday, August 15, 2016 10:14
> > >From: Jim web Sent: Sunday, August 14, 2016 16:48

> > >However the 'American in Paris' snippet file I fetched gives no sound
> > >at all from VLC! The video looks fine, but its a silent film! 8-)

> Does this have any relevance to your silent problem?
> https://squarepenguin.co.uk/forums/thread-988.html The title of the
> thread is loss of audio on conversion of video when using mode hvf 


Yes, that may be what's happening. I am using a relatively old LTS version
of Mint. I don't want to upgrade distro juts for this, so I'll see if I can
update either VLC or avconv.

I've also raised this particular issue on a linux newsgroup and had some
suggested fixes there. So I'll look at those as well.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: David Cantrell Sent: Monday, August 15, 2016 23:37


I'd like to rate-limit get_iplayer so that other things on the same machine 
that are also talking to the internet don't run like stunned slugs in 
treacle.



Second, I'm not using Linux.


You don't say which non-Linux operating system you are using.  Your 
description doesn't correspond with any version of Windows, 32 bit or 64 
bit, I have used except maybe Windows 3.1.  Here downloading HLSHD with 
get_iplayer at speeds up to 80Mbit/s on an elderly 2.33GHz Core 2 Duo with 
4GByte of RAM in 64 bit Windows 10 Home and a 10/100 LAN card gave a CPU 
usage of between 20% and 52% and memory usage barely above background.  How 
much RAM have you got?





___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Monday, August 15, 2016 10:14
>From: Jim web Sent: Sunday, August 14, 2016 16:48



>However the 'American in Paris' snippet file I fetched gives no sound
>at all from VLC! The video looks fine, but its a silent film! 8-)


Does this have any relevance to your silent problem?
https://squarepenguin.co.uk/forums/thread-988.html
The title of the thread is loss of audio on conversion of video when using 
mode hvf 




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

Had an email this morning. Just to say that, with some luck, we may find
that the missing segments problem could be cured soon. Fingers crossed...

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <887A76E5507D40D498F63F902B02CA7A@RJCDESK>, RS
 wrote:
> >From: Jim web Sent: Monday, August 15, 2016 17:59

> >This morning I used the 'best' option, but it now occurs to me: Does
> >that no longer use the 'flash' mode? I just tried a couple of fetches
> >explicitly specifying 'flash' and they fetched OK, as I was getting
> >until the last few days.

> I think what has caused confusion is that the transition from v2.94 to
> v2.95 made several changes at the same time.

> HLS  and HVF support were added. The default was changed from Flash to
> HLS. The default was changed from better to best. Some of the changes
> such as HLS support may have been in the development version you used
> while others were not.

Yes. My apologies for missing this change. It meant that what I wrote about
the 'flash mode' results yesterday was wrong.

> To be certain of what you are testing you need to specify the mode
> explicitly such as --tvmode=flashhd or --tvmode=flashvhigh or
> --tvmode=hvfvhigh or --tvmode=hlshd

That raises a question about the optimum fallback when a mode isn't
available for a given file.

In some cases when I tried 'hvfhd' I was told that wasn't available. (This
is for some of the 'section' files from Proms.) I was then wondering about
what 'sequence' of modes would be best for finding the 'best available'
version. (And also dodging problems like the current ones.)

e.g. having something like  --tvmode=hvfhd1,hvfhd2,hlshd,...

I've not yet tried the lower res hvf so don't know how it compares with the
others.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
,
   tellyaddict  wrote:
> The GiP output will also tell you which mode it has used to download.
> "INFO: Trying flashvhigh1 mode to record tv: Britain on Film: Series 1 -
> 2. Brits at Play"

Noted. Athough I confess that one problem I have is that the output from
gip is *too* verbose for me! When using my front end the reports all appear
in a terminal. I had to extend the length of the terminal window's text
store to 800 lines this morning as the old limit I'd set (300 lines) was
overrun when I fetched 9 items! Maybe I sould redirect a copy to a file for
later examination.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
,
   tellyaddict  wrote:
> The sync problems are caused by converting a programme with missing
> segments after download. --raw files have no sync issues even if there
> are segments missing.

I've never tried the --raw option. Might it help with the following?

A number of the files I fetched yesterday morning had reported missing
segments. This specified time offsets in secs. I had a look at those
points.

In some cases there was no sign of a problem. In others the video froze
for, say, ten sec, and when it resumed the video and audio seemed still in
synch. But in other cases the synch was out.

Would the synch offset effect be avoided by using --raw?

If so, what other problems might that give? My first thought might be that
the result would play with something like VLC without any timing. i.e. the
total and elapsed times would be zero and attempts to jump to some point
could cause problems with replay.

My assumption is that the pass though ffmpeg at the end of a fetch is to
tidy up these issues. (It is something I have to do with ts files grabbed
from DVB-T2.) But I am assuming/guessing this.

BTW I did a refetch this morning. The files I had segment problems with
yesterday I got using the modes=flash option and seem fine.

I've also tried mplayer on the hvfhd versions and that fails to find the
audio on some of them, just like vlc!

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: RE: Audio/Video Out of Sync

[tellyaddict]

I don't think it's speed that's the issue. You can try downloading the failed 
segments in a browser and they fail because they are just not on the server. 
Working segments (eg. the one before the failure) will download fine.

__

> Having a thought about this, when you stream something through IPlayer or on
> a device of any sort to watch as it streams, it pulls data at a far lower
> rate than when we try and download the whole programme in one hit. I wonder
> whether it is this attempt to get the data quickly that is causing the
> problem. If I were wearing a tin hat I would say it could have been a
> deliberate change to cause problems to people using get_iplayer :-)
> 
> One way to test this would be to make get_iplayer behave more like a
> "legitimate" (for want of a better word) client and pull the data at closer
> to real time speeds. May be worth doing as a test if someone has the
> time/inclination.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[David Cantrell]

On 2016-08-15, 18:08, John Bingham wrote:

On 15/08/16 17:42, David Cantrell wrote:

I think that's unlikely. That said, if someone were to add rate-limiting
options to get_iplayer I'd be very happy.

Fairly easy to slow down everything on a Linux interface.  If anyone has
the time/inclination, maybe worth investigating the tc command.


Two things.

First, slowing down *an entire interface* is exactly what I don't want. 
I'd like to rate-limit get_iplayer so that other things on the same 
machine that are also talking to the internet don't run like stunned 
slugs in treacle.


Second, I'm not using Linux.

--
David Cantrell | Minister for Arbitrary Justice

  Longum iter est per praecepta, breve et efficax per exempla.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: George Eycott
Sent: Monday, August 15, 2016 15:52


Having a thought about this, when you stream something through IPlayer or 
on

a device of any sort to watch as it streams, it pulls data at a far lower
rate than when we try and download the whole programme in one hit. I wonder
whether it is this attempt to get the data quickly that is causing the
problem.


The errors are repeatable, so it seems unlikely that they are being caused 
by server loading  or by downloading too fast.  If you look at dinky's 
pre-Olympics list

https://squarepenguin.co.uk/forums/thread-979-post-4398.html#pid4398
you will see that some of them have been fixed.

Anyway get_iplayer's HLS download can be slowed in Windows 10 Home (but not 
Pro) by sending the --output to a FAT32 drive.  The larger the drive the 
greater the slowing.  I downloaded Versailles episode 9 (b07n7dbg) in HLSHD 
at 65Mbit/s, 2.5Mbit/s and 16.6Mbit/s.  All three attempts found a Segment 
not found error at segment 52.




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Alan Milewczyk]

On 15/08/16 22:07, RS wrote:

S
I think what has caused confusion is that the transition from v2.94 to 
v2.95

made several changes at the same time.


Agreed


HLS  and HVF support were added.


HLS was already available in v2.94, I've been using it for ages, ever 
since I found that I could download large files (like "Wimbledon") 
without hitting the 4GB RTMP problem.

The default was changed from Flash to HLS.
The default was changed from better to best.
Some of the changes such as HLS support may have been in the development
version you used while others were not.

To be certain of what you are testing you need to specify the mode
explicitly such as
--tvmode=flashhd or --tvmode=flashvhigh or --tvmode=hvfvhigh
or --tvmode=hlshd


The rest is true.

A

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

Sorry I sent this to the wrong address.


From: Jim web
Sent: Monday, August 15, 2016 17:59



This morning I used the 'best' option, but it now occurs to me: Does that
no longer use the 'flash' mode? I just tried a couple of fetches explicitly
specifying 'flash' and they fetched OK, as I was getting until the last few
days.


I think what has caused confusion is that the transition from v2.94 to v2.95
made several changes at the same time.

HLS  and HVF support were added.
The default was changed from Flash to HLS.
The default was changed from better to best.
Some of the changes such as HLS support may have been in the development
version you used while others were not.

To be certain of what you are testing you need to specify the mode
explicitly such as
--tvmode=flashhd or --tvmode=flashvhigh or --tvmode=hvfvhigh
or --tvmode=hlshd 




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Monday, August 15, 2016 17:59



This morning I used the 'best' option, but it now occurs to me: Does that
no longer use the 'flash' mode? I just tried a couple of fetches explicitly
specifying 'flash' and they fetched OK, as I was getting until the last few
days.


I think what has caused confusion is that the transition from v2.94 to v2.95 
made several changes at the same time.


HLS  and HVF support were added.
The default was changed from Flash to HLS.
The default was changed from better to best.
Some of the changes such as HLS support may have been in the development 
version you used while others were not.


To be certain of what you are testing you need to specify the mode 
explicitly such as
--tvmode=flashhd or --tvmode=flashvhigh or --tvmode=hvfvhigh 
or --tvmode=hlshd




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

The sync problems are caused by converting a programme with missing segments 
after download. --raw files have no sync issues even if there are segments 
missing.

> > I'm still using V2.94 and no sync problems.
> >
> > Is the audio out by a constant time (eg 0.3 secs) in a programme?
> >
> > I haven't been following this thread.
> >
> > _
> 
> No, the download drops segments and the sync difference relates to the 
> number of segments missing. Some programmes just have one segment, 
> others much more, I've seen time differences of more than a minute.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Peter S Kirk]

On 15 Aug 2016 at 18:35, tellyaddict tellyaddict  
wrote:

> > OK, I think this thread has now ran it's course. Enough of the
> > speculation and OT chatter.
> 
> I have to disagree.
> 
> I don't think this is OT. This is a current problem with the current version 
> of GiP that we are trying to work out.
> 
> > The workaround for most users is to use the old flash streams --modes=flash

I'm still using V2.94 and no sync problems.

Is the audio out by a constant time (eg 0.3 secs) in a programme?

I haven't been following this thread.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[batguano999]

 >  
 > I think so far we have worked out quite a lot about the problems ...
 > 
 > 

Hi
Those hls streams give problems with "some" radio shows too...
=
=

Flash downloads OK...
$ get_iplayer --force --mode=flashstd --stream --type=radio 
--url=http://www.bbc.co.uk/programmes/b07nn86s | \
ffmpeg -y -i - -ac 1 -ar 16000 -sample_fmt s16 -map_metadata -1 
flashstd_flac.flac

Output #0, flac, to 'flashstd_flac.flac':
  Metadata:
encoder : Lavf57.46.101
Stream #0:0: Audio: flac, 16000 Hz, mono, s16, 128 kb/s
Metadata:
  encoder : Lavc57.51.102 flac
Stream mapping:
  Stream #0:0 -> #0:0 (aac (native) -> flac (native))
180355.014 kB / 10995.18 sec (99.9%)size=  218872kB time=03:03:10.36 bitrate= 
163.1kbits/s speed=30.6x
Download complete

INFO: Streaming completed successfully
size=  218961kB time=03:03:15.20 bitrate= 163.1kbits/s speed=30.6x
video:0kB audio:218953kB subtitle:0kB other streams:0kB global headers:0kB 
muxing overhead: 0.003697%
==

But hls throws out garbage (sometimes)...
$ get_iplayer --force --mode=hlsstd --stream --type=radio 
--url=http://www.bbc.co.uk/programmes/b07nn86s | \
ffmpeg -y -i - -ac 1 -ar 16000 -sample_fmt s16 -map_metadata -1 hlsstd_flac.flac

Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] channel element 2.14 is not allocated
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] channel element 3.8 is not allocated
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] Reserved bit set.
[aac @ 0xacb29c0] Prediction is not allowed in AAC-LC.
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] channel element 3.7 is not allocated
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] Sample rate index in program config element does not match 
the sample rate index configured by the container.
[aac @ 0xacb29c0] Remapped id too large
[aac @ 0xacb29c0]  is not implemented. Update your FFmpeg version to the newest 
one from Git. If the problem still occurs, it means that your file has a 
feature which has not been implemented.
[aac @ 0xacb29c0] If you want to help, upload a sample of this file to 
ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. 
(ffmpeg-de...@ffmpeg.org)
Error while decoding stream #0:0: Not yet implemented in FFmpeg, patches welcome
[aac @ 0xacb29c0] Number of scalefactor bands in group (15) exceeds limit (14).
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] Reserved bit set.
[aac @ 0xacb29c0] Prediction is not allowed in AAC-LC.
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] channel element 2.8 is not allocated
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] channel element 2.1 is not allocated
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] Prediction is not allowed in AAC-LC.
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] Number of bands (20) exceeds limit (4).
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] channel element 3.9 is not allocated
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] Number of bands (7) exceeds limit (1).
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0xacb29c0] Sample rate index in program config element does not match 
the sample rate index configured by the container.
[aac @ 0xacb29c0] Inconsistent channel configuration.
[aac @ 0xacb29c0] get_buffer() failed
Error while decoding stream #0:0: Invalid argum
exit
==

And sometimes hls is OK...
$ get_iplayer --force --mode=hlsstd --stream --type=radio 
--url=http://www.bbc.co.uk/programmes/b07nnbjr | \
ffmpeg -y -i - -ac 1 -ar 16000 -sample_fmt s16 -map_metadata -1 
hlsstd_flac2.flac

Output #0, flac, to 'hlsstd_flac2.flac':
  Metadata:
encoder : Lavf57.46.101
Stream #0:0: Audio: flac, 16000 Hz, mono, s16, 128 kb/s
Metadata:
  encoder : Lavc57.51.102 flacINFO: Streaming completed successfully
size=2745kB time=00:02:13.52 bitrate= 168.4kbits/s speed=29.2x
video:0kB audio:2737kB subtitle:0kB other streams:0kB global headers:0kB muxing 
overhead: 0.295743%
=

Re: Audio/Video Out of Sync

[tellyaddict]

> Not much point in going much further on that point. But over the years I
> have learned that speculations about the cause or reason for something are
> often helpful in being able to get something sorted out. So I don't agree
> with you. We''ll have to differ in our views on this.

I think so far we have worked out quite a lot about the problems through this 
"meaningless noise" as S Carr puts it.

>S Carr  wrote:
> > Hats off to dinky for the work, but please realise get_iplayer is a
> > hack, a workaround, call it whatever you want, it's not a legitimate way
> > to "consume" iPlayer content, those apps the BBC produce will have a lot
> > more error checking and recovery in them than get_iplayer does, so all
> > this speculation on how things work is just meaningless noise.

You don't have to be involved if you don't want to be.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Alan Milewczyk]

On 15/08/16 17:59, Jim web wrote:


Until a few days ago I was using 'best' and I think this was giving me the
'RTMPDump'. *However* a few days ago I changed from using a 'development'
version of 2.95 to the 2.95 released a while ago. At that point, and until
today, I've been focussed on trying to diagnose the hls problems and,
latterly, on the puzzles with the hvf files lacking audio when played.

This morning I used the 'best' option, but it now occurs to me: Does that
no longer use the 'flash' mode? I just tried a couple of fetches explicitly
specifying 'flash' and they fetched OK, as I was getting until the last few
days. So my apologies if I overlooked this and and in fact the flash mode
is still mostly working OK. It was because I was using my program that
chooses between predefined sets of options and I've forgotten it used
'best' for the one in question.


If I read this correctly then you're confusing two things, the mode of 
download (HLS, HVF, FLASH) and the quality level (best, hd, vhigh, etc).


To quote from the release notes (2. New default streaming formats):


   "TV

 * HLS (HTTP Live Streaming) is now the default format for on-demand TV
 * HLS downloads are MPEG-TS (transport stream) files, which can be
   played by most media players. The files are re-muxed to MP4 by
   default to make them palatable to AtomicParsley for metadata
   tagging. Downloading with|--raw|will produce MPEG-TS files.
 * RTMP (Flash) format is temporarily still available
   with|--tvmode=flash|. The previous default setting is available
   with|--tvmode=flashbetter|."

And from the Wiki "Modes"


   "Quality levels

Quality level can be one of five possible values (in order of decreasing 
bitrate and resolution):


 * best
 * better
 * good
 * worse
 * worst"


   "The
   stream type prefixes and quality levels can be combined,
   e.g.|hlsgood|,|hvfbest|,|dashworst|,"


HTH


Alan




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[John Bingham]

Fairly easy to slow down everything on a Linux interface.  If anyone has 
the time/inclination, maybe worth investigating the tc command.


On 15/08/16 17:42, David Cantrell wrote:

On Mon, Aug 15, 2016 at 03:52:52PM +0100, George Eycott wrote:


Having a thought about this, when you stream something through IPlayer or on
a device of any sort to watch as it streams, it pulls data at a far lower
rate than when we try and download the whole programme in one hit. I wonder
whether it is this attempt to get the data quickly that is causing the
problem. If I were wearing a tin hat I would say it could have been a
deliberate change to cause problems to people using get_iplayer :-)


I think that's unlikely. That said, if someone were to add rate-limiting
options to get_iplayer I'd be very happy.



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
, S
Carr  wrote:
> On 15 August 2016 at 16:33, Jim web  wrote:
> > If your 'Flash' method differs from mine, please specify the details
> > and I'll try it. But at present I am spending over an hour per morning
> > fetching files I then mostly get problems with. From what others have
> > written I don't seem to be the only person encountering problems.

> I previously used --modes=best, switched that to --modes=flash not seen
> any problems since.

I must add a question/apology/comment which came to me after I sent my
previous email!

Until a few days ago I was using 'best' and I think this was giving me the
'RTMPDump'. *However* a few days ago I changed from using a 'development'
version of 2.95 to the 2.95 released a while ago. At that point, and until
today, I've been focussed on trying to diagnose the hls problems and,
latterly, on the puzzles with the hvf files lacking audio when played.

This morning I used the 'best' option, but it now occurs to me: Does that
no longer use the 'flash' mode? I just tried a couple of fetches explicitly
specifying 'flash' and they fetched OK, as I was getting until the last few
days. So my apologies if I overlooked this and and in fact the flash mode
is still mostly working OK. It was because I was using my program that
chooses between predefined sets of options and I've forgotten it used
'best' for the one in question.

This does leave us with the other problems, though.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article

Re: Audio/Video Out of Sync

[David Cantrell]

On Mon, Aug 15, 2016 at 03:52:52PM +0100, George Eycott wrote:

> Having a thought about this, when you stream something through IPlayer or on
> a device of any sort to watch as it streams, it pulls data at a far lower
> rate than when we try and download the whole programme in one hit. I wonder
> whether it is this attempt to get the data quickly that is causing the
> problem. If I were wearing a tin hat I would say it could have been a
> deliberate change to cause problems to people using get_iplayer :-)

I think that's unlikely. That said, if someone were to add rate-limiting
options to get_iplayer I'd be very happy.

-- 
David Cantrell | A machine for turning tea into grumpiness

[OS X] appeals to me as a monk, a user, a compiler-of-apps, a
sometime coder, and an easily amused primate with a penchant
for those that are pretty, colorful, and make nice noises.
-- Dan Birchall, in The Monastery

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[S Carr]

On 15 August 2016 at 17:35, tellyaddict  wrote:
> We have been told for a long time that the Flash streams will disappear. Not 
> sure this is really the answer.

Being told one thing and it actually happening are not the same. When
they disappear then I'll worry about it. Using flash might not be the
ultimate fix to the issue but it got me back up and working.

Hats off to dinky for the work, but please realise get_iplayer is a
hack, a workaround, call it whatever you want, it's not a legitimate
way to "consume" iPlayer content, those apps the BBC produce will have
a lot more error checking and recovery in them than get_iplayer does,
so all this speculation on how things work is just meaningless noise.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

> But they want people to shift away from using Akamai, so may not be
> concerned. I don't use the standard BBC webpage interfaces, etc, these days
> so have no idea if current browsers or BBC apps are being affected by this.
> If they are, I'd have expected an outcry by now as this has been happening
> for weeks.

Browsers use HDS so probably aren't affected. They are served by both Akamai 
and Limelight so are probably on the "current" system so the BBC are probably 
packaging these streams themselves. I think iPads etc now use the HVF streams 
so again probably are not affected.

Akamai is still a major part of iPlayer. Between it and Limelight they serve 
all the HDS, HVF (current HLS) and Flash streams.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

> OK, I think this thread has now ran it's course. Enough of the
> speculation and OT chatter.

I have to disagree.

I don't think this is OT. This is a current problem with the current version of 
GiP that we are trying to work out.

> The workaround for most users is to use the old flash streams --modes=flash

We have been told for a long time that the Flash streams will disappear. Not 
sure this is really the answer.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[S Carr]

On 15 August 2016 at 16:33, Jim web  wrote:
> If your 'Flash' method differs from mine, please specify the details and
> I'll try it. But at present I am spending over an hour per morning fetching
> files I then mostly get problems with. From what others have written I
> don't seem to be the only person encountering problems.

I previously used --modes=best, switched that to --modes=flash not
seen any problems since.

Options I have set are: --versions=editorial,default --fatfilename --whitespace

Steve

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
, S
Carr  wrote:
> OK, I think this thread has now ran it's course. Enough of the
> speculation and OT chatter.

Most of this seems very on topic for this list to me. cf below in case you
have missed this...

> The workaround for most users is to use the old flash streams
> --modes=flash

Is it? I fetched about 10 items this morning using the old 'Flash RTMPDump'
method. Most of them show problems due to 'missing segments'. I've been
working though them finding sections where the video freezes for up to 10
sec and, when video resumes, the synch is clearly well out. (Note the title
of this thread if you want to complain about this being 'OT'.)

This is from some Proms during the last week.

So as things stand I get:

--mode=best is the oldest method. Until a few days ago that seemed OK. But
not now. It gives me the above problems.

--tvmode=hlsbest is faster and avoided glitches - when it works - but tends
to end prematurely. (And repeatedly does so on retrys.)

and now the hvfhd method tends to give me files where I get no audio.
ffmpeg tells me that these files are malformed. Although of course these
problems may be down to ffmpeg and vlc (and now mplayer which also can't
find the audio on these files when I installed and tried it.)

If your 'Flash' method differs from mine, please specify the details and
I'll try it. But at present I am spending over an hour per morning fetching
files I then mostly get problems with. From what others have written I
don't seem to be the only person encountering problems.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[S Carr]

OK, I think this thread has now ran it's course. Enough of the
speculation and OT chatter.

The workaround for most users is to use the old flash streams --modes=flash

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
,
   tellyaddict  wrote:
> > I can only speculate here, but: It might be the case that some of
> > Akamai's non-BBC 'clients' are wanting the changes for some reasons of
> > their own.

> Why would anyone ask for broken streams though?

I can't tell, I'm speculating. But they may have been changed in a specific
way that their clients expect and deal with, and gives them something
useful. If the changes are expected and planned by somebody and they are
handling them, then they may not regard this as 'broken' at all.

> > Or they stem from an arrangement the BBC aren't party to. They might
> > be dealing with it in some way which the BBC aren't aware of.

> The BBC would be quite a big client to upset.

But they want people to shift away from using Akamai, so may not be
concerned. I don't use the standard BBC webpage interfaces, etc, these days
so have no idea if current browsers or BBC apps are being affected by this.
If they are, I'd have expected an outcry by now as this has been happening
for weeks.

I have no idea if any of the above is the case. If it isn't then we have to
reverse the questions and ask why Akamai clients aren't all pounding on
Akamai and forcing them to return to business as usual? If they *are* then
Akamai do seem a little slow in dealing with this. If so, that won't help
their reputation with future potential clients.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: RE: Audio/Video Out of Sync

[tellyaddict]

> I can only speculate here, but: It might be the case that some of Akamai's
> non-BBC 'clients' are wanting the changes for some reasons of their own.

Why would anyone ask for broken streams though?

> Or they stem from an arrangement the BBC aren't party to. They might be
> dealing with it in some way which the BBC aren't aware of.

The BBC would be quite a big client to upset.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <01d1f6e4$f74537a0$e5cfa6e0$@macfh.co.uk>, MacFH - C E
Macfarlane  wrote:
> > 
> > It may only be depreciated as far as the BBC is concerned. Although the
> BBC
> > is now moving towards packaging its own content, Akamai's other
> > clients might continue to use the service.

> That's a good point.  I hadn't thought about other clients.  Thinking
> about it, if Akamai's current problems are affecting every customer, 
> they must be causing so much grief that I'm surprised that they haven't
> simply rolled back the offending 'upgrade' and reinstated their former
> system.  In fact, given the problems with BBC content, I'm surprised
> that they haven't done that anyway, so presumably there must be a fairly
> compelling reason why not.

I can only speculate here, but: It might be the case that some of Akamai's
non-BBC 'clients' are wanting the changes for some reasons of their own. Or
they stem from an arrangement the BBC aren't party to. They might be
dealing with it in some way which the BBC aren't aware of.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

RE: Audio/Video Out of Sync

[MacFH - C E Macfarlane]

Please see below ...

> -Original Message-
> From: get_iplayer [mailto:get_iplayer-boun...@lists.infradead.org] On
> Behalf Of tellyaddict
> Sent: 14 August 2016 14:44
> To: get_iplayer
> Subject: Re: Audio/Video Out of Sync
> 
> It may only be depreciated as far as the BBC is concerned. Although the
BBC
> is now moving towards packaging its own content, Akamai's other clients
> might continue to use the service.

That's a good point.  I hadn't thought about other clients.  Thinking about
it, if Akamai's current problems are affecting every customer,  they must be
causing so much grief that I'm surprised that they haven't simply rolled
back the offending 'upgrade' and reinstated their former system.  In fact,
given the problems with BBC content, I'm surprised that they haven't done
that anyway, so presumably there must be a fairly compelling reason why not.


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <6688868477144804BAF7EFA66BCD19BD@RJCDESK>, RS
 wrote:
> >From: Jim web Sent: Sunday, August 14, 2016 16:48

> >However the 'American in Paris' snippet file I fetched gives no sound
> >at all from VLC! The video looks fine, but its a silent film! 8-)

> An American in Paris (p044lmdc) plays fine here with VLC.  I used
> --tvmode hvfhd and it downloaded hvfhd1.  It is 14min58s and 553MByte. 
> Unlike the broadcast, there is only one stereo sound track.

I also used hvfhd1 mode but via

--modes=hvfhd

but I assume that's a synonym for using --tvmode here (?)

And yes, I've used ffprobe to check. The files all seem to have just one
stereo audio aac track. The good news there is they are higher bitrate than
the hlshd versions.

Another curio is that yesterday I started fetching the hvfhd version of the
Wilson/Gershwin prom but ctrl-C'd it after a few mins because I didn't want
over 4 MG more download during the daytime. That snippet *did* play out
audio with VLC.

However I fetched the entire file this morning before 9am. This file
*doesn't* play any audio when I give it to VLC. But again, ffprobe says the
audio is there.

Like others, I wish there was a 25pfs 1280x720 hvfhd. Indeed, to me for
things like proms it would make more sense to use 25fps with a 320k stereo
aac audio stream!

I suspect the problem I am getting is something about the version of VLC I
am using or its settings. So I'll raise that on a Linux group in case
someone there can twig it.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

The HVF streams are still HLS. The HLS streams that are being classed as legacy 
though sadly are the 25fps equivalents.

__

> It says,"A key to making this work was a switch to chunked HTTP video 
> formats. We already use these for our live streams and with this change all 
> new content will only be available via chunked HTTP distribution. That is 
> HDS (HTTP Dynamic Streaming), to replace the current RTMP for Flash based 
> players, HLS (HTTP Live Streaming) for iOS and other connected devices."
> 
> There is no suggestion there that HLS is a legacy product.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: RS Sent: Sunday, August 14, 2016 23:31



There is also a link to a blog.


The blog I referred to is well worth a read, although it's a year old.
http://www.bbc.co.uk/blogs/internet/entries/e2428e08-7e8b-4617-8d39-244e482b8895

It says,"A key to making this work was a switch to chunked HTTP video 
formats. We already use these for our live streams and with this change all 
new content will only be available via chunked HTTP distribution. That is 
HDS (HTTP Dynamic Streaming), to replace the current RTMP for Flash based 
players, HLS (HTTP Live Streaming) for iOS and other connected devices."


There is no suggestion there that HLS is a legacy product.




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: RS Sent: Sunday, August 14, 2016 22:53


If we had a 25fps equivalent of HVFHD we wouldn't need HLSHD or FlashHD. 
There are two modes at 960x540.  HVFSD is 50fps and HVFvhigh is 25fps.  A 
mode at 1280x720 25fps is the obvious step down from HVFHD, but it isn't 
there.  It's strange since a lot of content doesn't need to be 50fps.


I've just found some articles by someone called Christopher Woods entitled 
How good is HD supposed to be.  There is a link to

http://www.bbc.co.uk/rd/blog/2015-07-the-development-of-new-video-factory-profiles-for-bbc-iplayer
which gives some information on the Video Factory project.
"At around 3 Mbit/s a 960x540 profile at 50fps will be made available to 
Connected TVs and set top boxes. The Elemental encoders are used to convert 
the 1920x1080 interlaced content to 960x540 for progressive encoding at 
50fps. Although the 960x540 profile has a reduced spatial resolution 
compared to the outgoing 1280x720 at 25fps, subjective assessments shows it 
delivers significantly better pictures on TV screens across a wide range of 
popular content (such as EastEnders and Top Gear) due to its higher frame 
rate. The 50fps, 1280x720 profile, however, will be available to those with 
5Mbit/s broadband connections."


There is also a link to a blog.




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Sunday, August 14, 2016 14:54



But in the end I assume the correct response is to cease needing the
'deprecated' old arrangements anyway.


If we had a 25fps equivalent of HVFHD we wouldn't need HLSHD or FlashHD. 
There are two modes at 960x540.  HVFSD is 50fps and HVFvhigh is 25fps.  A 
mode at 1280x720 25fps is the obvious step down from HVFHD, but it isn't 
there.  It's strange since a lot of content doesn't need to be 50fps. 




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Sunday, August 14, 2016 16:48



However the 'American in Paris' snippet file I fetched gives no sound at
all from VLC! The video looks fine, but its a silent film! 8-)


An American in Paris (p044lmdc) plays fine here with VLC.  I used --tvmode 
hvfhd and it downloaded hvfhd1.  It is 14min58s and 553MByte.  Unlike the 
broadcast, there is only one stereo sound track.





___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Alan Milewczyk]

On 14/08/16 16:48, Jim web wrote:


Anyone explain? These files are a bit big to keep trying during the
daytime! Although the fetching seems pretty fast, they'd eat up my quota if
I had to keep re-fetching. So a more targetted approach to diagnosing this
would be good. Is the file weird, or is VLC the problem, or... ?

Jim


Have you checked if there is more than one audio track?

Alan


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

A new puzzle springs out at me as I try to negotiate the current
'segmentation' problems.

I decided to try the hvfhd1 mode and chose some of the sections of the
Wilson / Gershwin Prom check this out.

The 'Rhapsody in Blue' snippet file the BBC offer is fine. 50fps 1280x720
with a 125k stereo soundtrack. Plays fine with VLC (my usual player).

I also started fetching the full Prom, but ctrl-C'd it after a while as I
just wanted to check using the first few mins. This also plays fine.

However the 'American in Paris' snippet file I fetched gives no sound at
all from VLC! The video looks fine, but its a silent film! 8-)

ffmprobe says it is the same as the Rhapsody. And if  drop it onto
Audacious *that* can play the audio in the mp4 without any problems. But
VLC is silent.

So Audacious can play the audio from all the above. ffmprobe can show me
the details which seem the same. But two play with audio as usual with VLC
whilst the other stays silent.

I used

 --modes=hvfhd1 --no-tag pid <>

for each. (I also tried omitting the no-tag.)

Anyone explain? These files are a bit big to keep trying during the
daytime! Although the fetching seems pretty fast, they'd eat up my quota if
I had to keep re-fetching. So a more targetted approach to diagnosing this
would be good. Is the file weird, or is VLC the problem, or... ?

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

Another thing to remember is that both Limelight and Akamai are serving the 
dashhigh streams, so it could be that 1 is behaving better than the other. I 
might have got lucky yesterday and hit the right server and you might have got 
unlucky this morning and hit the troublesome one and got the right one this 
afternoon.
_

> Yes. I tried it again this morning and it failed again.
> 
> However I tried it again having seen you email, and this time it worked OK!

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <000101d1f62e$5c323da0$1496b8e0$@macfh.co.uk>, MacFH - C E
Macfarlane  wrote:

> ... and, as Jim has said, it's a legacy system, so why would they be
> changing it?  It's not impossible of course, but doesn't seem likely if
> it's deprecated and due to be discontinued anyway.

It is possible that a part of the problem is that the BBC aren't doing the
segmenting, etc, for Akamai, and can't directly check what is being done on
it. Thus some recent change might be the cause, but the BBC won't know what
the change has been.

The puzzle is why we get inconsistencies in which items are affected, and
why some later 'recover' whilst others don't. It may be someone is simply
dealing 'by hand' with problems they notice have time to deal with, but
haven't dealt (or found) the cause.

But in the end I assume the correct response is to cease needing the
'deprecated' old arrangements anyway.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
,
   tellyaddict  wrote:
> That's not a good sign if these errors are starting to creep into DASH
> as well. DASH is on the "current" system so shouldn't have problems.

> Is it PID b07mz0hj that you were downloading? I've just tried it and it
> downloaded without showing any errors for me in dashhigh.

Yes. I tried it again this morning and it failed again.

However I tried it again having seen you email, and this time it worked OK!

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

> ... and, as Jim has said, it's a legacy system, so why would they be
> changing it?  It's not impossible of course, but doesn't seem likely if it's
> deprecated and due to be discontinued anyway.

It may only be depreciated as far as the BBC is concerned. Although the BBC is 
now moving towards packaging its own content, Akamai's other clients might 
continue to use the service.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

It could be that when a missing segment is found, the servers log it as an 
error. Then when they review things later, they go back and correct what's been 
flagged up. As you say it could also be reports from the BBC and users that 
causes Akamai to go back and fix it.

As for why things have got steadily worse, it could be that a lot of the 
content was processed in advance of broadcast so may have been processed while 
the system worked. Now that we are a month on, we might be hitting programmes 
that have largely been processed since the update (if that's what caused the 
problems in the first place).

__

> I agree that one question is what happened a month ago to change things, and 
> it could have been a buggy software update.  However The Moth in the Flame 
> programme (b07l1wv4) got corrected a week after broadcast.  It could be that 
> it was edited and re-encoded as a result of the edit.  It could also be that 
> the Segment not found error was brought to the attention of the BBC or 
> Akamai and that was why it was corrected.  The second question is why things 
> have got steadily worse over the intervening month, so that now virtually 
> all programmes are affected.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

RE: RE: Audio/Video Out of Sync

[MacFH - C E Macfarlane]

Thanks for your reply, please see below ...

> -Original Message-
> From: get_iplayer [mailto:get_iplayer-boun...@lists.infradead.org] On
> Behalf Of tellyaddict
> Sent: 13 August 2016 21:51
> To: get_iplayer
> Subject: Re: RE: Audio/Video Out of Sync
> 
> > That successfully shoots down my tentative explanation, but
> > unfortunately doesn't supply a better one.  Based on your superior
> > knowledge of both GiP, and the BBC system, what is your understanding
> > of why these problems started to occur about a month ago, bearing in
> > mind that I've seen at least one in a DASH download as well (the night
> > before last, last night's downloads were error free).

Just FYI, all overnight downloads have been error-free since that one.

> I wouldn't really say that I have any superior knowledge on this. My best
> guess on why these problems suddenly started about a month ago would
> be...
> All of the segmenting, playlist creation etc will be done by some software
at
> Akamai if they are doing the packaging. It's possible that maybe they have
> done a software update that has introduced a bug into their encoding
> systems.

Yes, but ...

> From what Jim said it sounds like Akamai have been doing this for
> the BBC for quite a while and in the past their software has processed
things
> without any problems.

... and, as Jim has said, it's a legacy system, so why would they be
changing it?  It's not impossible of course, but doesn't seem likely if it's
deprecated and due to be discontinued anyway.

Briefly, I'll go into more detail than I have previously about my
experiences with this issue.  I am using v2.95 on Linux.  I first started to
notice problems around Wimbledon, when I had one Linux NAS doing my normal
overnight downloads of everything except the tennis, and the other NAS doing
the tennis full-time.  I noticed that if I had two HLS downloads going at
once, one on each NAS, then inevitably I seemed to get problems with one or
the other or both, whereas if the Wimbledon NAS was downloading HLS, but the
other was downloading non HLS, nearly always it seemed alright.  At first I
thought this was due to trying to download two things at once on my low ADSL
bandwidth, but a week or less later others started to post about similar
problems, so now I suspect that I was simply one of the first to encounter
the current problems.

Given the coincidence with Wimbledon and now the Olympics, I think you can
see why I thought server loading might be an issue.  However, I note that
Jim has said that loading on ALL (my caps) servers was below capacity, by
which I presume he meant the Akamai ones as well as the BBC ones, so we are
still left short of an adequate explanation.  The only thing further I can
think of is that the putative change on the Akamai servers which you suggest
may actually have been the introduction of extra capacity for Wimbledon and
the Olympics, and this change, designed to thwart problems before they could
occur, somehow introduced the current problems as collateral damage.  I
guess we'll never know, but it sure is frustrating for everybody.

Also, I'm unclear from the release notes if the deprecating of the RTMP and
HLS timeout options means they are currently being ignored, or are still
effective.

PS (can be ignored by most):  In case further detail is useful to anyone
working on this issue, I'll give the download options which, rightly or
wrongly, were/are being used:
Normal Download NAS goes through the four PVR search lists
previously mentioned, a sequence of the following options ...
Radio HQ:   type radio, radiomode dashhigh
TV HQ:  type tv, tvmode flashhd,hlshd
Radio LQ:   type radio
TV LQ:  type tv
... as well as any one-off PVR jobs that have been added, which for
radio would usually be at normal quality, while TV ones would often have the
following options:
type tv, tvmode
flashhd,hlshd,flashvhigh,hlsvhigh
Wimbledon NAS (has to be HLS because RTMPDump falls over downloading
files greater than 4GB):
type tv, tvmode hlsvhigh, no-tag, raw 
Additionally the following global options were and are still set (including
several that are now deprecated):
rtmpdump /opt/share/bin/rtmpdump
exclude pets,reversions,shorts,short
edition,strange hill high
excludecategory Sign
Zone,Learning,Primary,Secondary,Children's
hlsradioopts --timeout 100
hlstvopts --timeout 100
isodate 1
refreshexclude BBC Radio 1,BBC 1Xtra,BBC
Three,CBBC,CBeebies
 

Re: Audio/Video Out of Sync

[RS]

From: tellyaddict
Sent: Saturday, August 13, 2016 21:51


My best guess on why these problems suddenly started about a month ago 
would be...
All of the segmenting, playlist creation etc will be done by some software 
at Akamai if they are doing the packaging. It's possible that maybe they 
have >done a software update that has introduced a bug into their encoding 
systems. From what Jim said it sounds like Akamai have been doing this for 
the BBC >for quite a while and in the past their software has processed 
things without any problems. So something has changed in the last month or 
so which has >affected pretty much all of the content Akamai processes for 
the BBC. All I can think of is that something must have changed with the 
systems that are >processing things.


I agree that one question is what happened a month ago to change things, and 
it could have been a buggy software update.  However The Moth in the Flame 
programme (b07l1wv4) got corrected a week after broadcast.  It could be that 
it was edited and re-encoded as a result of the edit.  It could also be that 
the Segment not found error was brought to the attention of the BBC or 
Akamai and that was why it was corrected.  The second question is why things 
have got steadily worse over the intervening month, so that now virtually 
all programmes are affected.




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: tellyaddict
Sent: Saturday, August 13, 2016 21:31



HLSHD and FlashHD are both 1280x720 25fps
HVFHD is 1280x720 50fps, but it probably only applies to Red Button and
Olympic channels.


Since tellyaddict has quoted what I said about HVFHD, I ought to point out 
that Alan is quite right, I have got things the wrong way round about HVF. 
Many Red Button and Olympics programmes are only available in HVF, from 
HVFHD to HVFlow.  Most programmes on the permanent channels are available in 
HVF, HLS and Flash.


If you want 1280x720 you can use --tvmode=hlshd,flashhd,hvfhd
If you don't want to risk HLS errors you can leave out hlshd
If you don't want 50fps you can leave out hvfhd



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: RE: Audio/Video Out of Sync

[tellyaddict]

I wouldn't really say that I have any superior knowledge on this. My best guess 
on why these problems suddenly started about a month ago would be...
All of the segmenting, playlist creation etc will be done by some software at 
Akamai if they are doing the packaging. It's possible that maybe they have done 
a software update that has introduced a bug into their encoding systems. From 
what Jim said it sounds like Akamai have been doing this for the BBC for quite 
a while and in the past their software has processed things without any 
problems. So something has changed in the last month or so which has affected 
pretty much all of the content Akamai processes for the BBC. All I can think of 
is that something must have changed with the systems that are processing things.
__

> That successfully shoots down my tentative explanation, but unfortunately
> doesn't supply a better one.  Based on your superior knowledge of both GiP,
> and the BBC system, what is your understanding of why these problems started
> to occur about a month ago, bearing in mind that I've seen at least one in a
> DASH download as well (the night before last, last night's downloads were
> error free).

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[tellyaddict]

It depends what you want. As Richard has said, flashhd and hlshd are pretty 
much equal to eachother BUT we now know that HLS is on a legacy system and 
Flash is due to be phased out at some point.

All the HLS options run at 25fps but are only supplied by Akamai who (at least 
for the time being) aren't packaging them in any useable way.

The only options on the "current system" are the HVF modes. Most of these 
though will give you a 50fps video. In the case of hvfhd, you will end up with 
a download thats 1280x720 50fps. But that 50fps frame rate will make an HD 
download about double the size of the current flashhd streams which are only 
25fps. If you want something that is roughly the same file size as the flashhd 
streams and are happy to settle for something that isn't HD resolution, you can 
use hvfsd which will give you a 960x540 50fps download. This is what iPlayer 
currently uses when you stream in HD.

> >is there a list showing which of these give what actual resolutions and -
> >where relevant - what limit there is on 'supplier'?
> 
> >What are the optimum choices here if I want 1280x720? Or is this a matter
> >of experiment?
> 
> HLSHD and FlashHD are both 1280x720 25fps
> HVFHD is 1280x720 50fps, but it probably only applies to Red Button and 
> Olympic channels.
> 
> In v2.95 best will give you HLSHD if it is available.  As far as I know if 
> HLSHD is not available it will fall back to FlashHD, as you have found.
> 
> I'll leave it to someone else to say what is provided by which CDN.

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article , RS
 wrote:
> >From: Jim web Sent: Saturday, August 13, 2016 09:37

> >BTW This morning an mpeg-dash fetch of the 320k version of yesterday's
> >"More Or Less" (R4) threw up segment error reports and ffmpeg failed to
> >turn the partial file into a tidied-up version when asked by gip that
> >Audacious could play. However doing this 'by hand' in a terminal window
> >worked OK.

> I don't know why you would want to listen to More or Less at 320kbit/s. 

I do prefer the 320k. But my real concern here is with Proms and the chance
they will also be affected. Today or tomorrow I will be fetching (or at
least trying to) a number of these from last week. For these, glitch-free
320k is quite important for me.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Saturday, August 13, 2016 09:37



BTW This morning an mpeg-dash fetch of the 320k version of yesterday's
"More Or Less" (R4) threw up segment error reports and ffmpeg failed to
turn the partial file into a tidied-up version when asked by gip that
Audacious could play. However doing this 'by hand' in a terminal window
worked OK.


I don't know why you would want to listen to More or Less at 320kbit/s.  I 
am happy with the podcast, although it is very different, but not noticeably 
(to my ears) in sound quality.  If you mean the shortened version of the one 
on grammar schools, b07mz0hj, I have just downloaded that manually at 
dashhigh without problem.  The continuity announcer on the shortened version 
sounds as though she is in a swimming pool. 




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article , RS
 wrote:

[snip useful info, with thanks]

> >My main problem wrt experimenting is that I tend to aim at doing tv
> >fetching before 9am when also preparing breakfast. I avoid doing too
> >much of this during the day to avoid clocking up too much accumlated
> >data count against my 'cap'

> I have been meaning to ask since you last said this whether you have
> looked at your ISP's website recently to see if there is a package
> which better suits your needs.  Most ISPs will just take your money
> without telling you there is an unlimited package for the same or less
> money as you are paying. 

I'll ask him about this. I'm currently asking him about having more web and
ftp space, etc, anyway. I use a small specialist ISP and he is generally
helpful.

Another reason for the "morning run" has been a tendency for the connection
speed to slow down during the day and become very slow in the evenings.
When I do a speed test I tend to be told the downloads run at around
70megs. But iplayer fetches get far slower than this.

I also find getting things in the morning quite convenient - provided the
fetching works reliably!

BTW This morning an mpeg-dash fetch of the 320k version of yesterday's
"More Or Less" (R4) threw up segment error reports and ffmpeg failed to
turn the partial file into a tidied-up version when asked by gip that
Audacious could play. However doing this 'by hand' in a terminal window
worked OK. I only had time to play half the result and it sounded OK. But
this is a worry given that I plan to fetch a batch of Proms over the
weekend! :-/

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Alan Milewczyk]

Further to my last posting, the available modes are shown in this wiki 
in the section "Recording Mode Details":


https://github.com/get-iplayer/get_iplayer/wiki/modes

HTH

Alan



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Alan Milewczyk]

On 12/08/16 22:05, RS wrote:

From: Jim web
Sent: Friday, August 12, 2016 15:50
is there a list showing which of these give what actual resolutions 
and -

where relevant - what limit there is on 'supplier'?


What are the optimum choices here if I want 1280x720? Or is this a 
matter

of experiment?


HLSHD and FlashHD are both 1280x720 25fps
HVFHD is 1280x720 50fps, but it probably only applies to Red Button 
and Olympic channels.


In v2.95 best will give you HLSHD if it is available.  As far as I 
know if HLSHD is not available it will fall back to FlashHD, as you 
have found.




Not true, it's available on all channels. I've downloaded programmes 
such as Saturday Kitchen and Highlands using hvfhd. I suspect hvfhd is 
available when hlshd is available, otherwise it will fall back to 
hlsvhigh/hvfsd (not flashhd).


Typing:
get_iplayer --info -pid 
will advise what modes are available for a particular programme.

I've seen a list of the various modes somewhere, I'll post it when I 
find it.


Alan

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[RS]

From: Jim web
Sent: Friday, August 12, 2016 15:50
is there a list showing which of these give what actual resolutions and -
where relevant - what limit there is on 'supplier'?



What are the optimum choices here if I want 1280x720? Or is this a matter
of experiment?


HLSHD and FlashHD are both 1280x720 25fps
HVFHD is 1280x720 50fps, but it probably only applies to Red Button and 
Olympic channels.


In v2.95 best will give you HLSHD if it is available.  As far as I know if 
HLSHD is not available it will fall back to FlashHD, as you have found.


I'll leave it to someone else to say what is provided by which CDN.


My main problem wrt experimenting is that I tend to aim at doing tv
fetching before 9am when also preparing breakfast. I avoid doing too much
of this during the day to avoid clocking up too much accumlated data count
against my 'cap'


I have been meaning to ask since you last said this whether you have looked 
at your ISP's website recently to see if there is a package which better 
suits your needs.  Most ISPs will just take your money without telling you 
there is an unlimited package for the same or less money as you are paying. 




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article
,
   tellyaddict  wrote:
> Hi Jim, How many times did your first command fail?

The command strings I use with my simple programmable fetcher are:

 sprintf(commandstring,"%s --type=tv --mode=best  --no-tag --pid %s
--attempts=2 --output %s\n",giputliname,pid,videooutdirname);

and

sprintf(commandstring,"%s --tvmode=hlsbest  --no-tag --pid %s --force
--attempts=2 --output %s\n",giputliname,pid,videooutdirname);

for these two methods. (The first %s gives the location of the version of
gip I've set for it to use. The second obviously adds the pid from the file
the program works though to fetch a list of items.)

(FWIW there are some other modes which I use for radio and for thing like
the old exclusion of akamai.)

A short time ago I had another try.

This time the 'old' (i.e. non-HLS method) worked OK and I seem to have to
full programme at 1280x720. When reprocessing at the end of the fetch it
did throw up four time values but didn't specify any errors. A quick look
at the specified times didn't show any problems.

So I assume that there is a fair chance that a forced retry this morning
may have worked OK, but I can't be sure.

A failure (e.g. I tried the HLS and it ended after about 30 sec) can mean
it gives *no* output file, but saves into the record details as if it
worked, so any futher attempt has to be forced, or I delete the file that
lists what has been 'downloaded'.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: Audio/Video Out of Sync

[Jim web]

In article <4C29776002E74A67945930E970693176@RJCDESK>, RS
 wrote:
> For testing or if there are problems I prefer to specify the precise
> --tvmode I want HLSHD, HLSvhigh FlashHD, Flashvhigh HVFHD, HVFSD,
> HVFvhigh

is there a list showing which of these give what actual resolutions and -
where relevant - what limit there is on 'supplier'?

What are the optimum choices here if I want 1280x720? Or is this a matter
of experiment?

My main problem wrt experimenting is that I tend to aim at doing tv
fetching before 9am when also preparing breakfast. I avoid doing too much
of this during the day to avoid clocking up too much accumlated data count
against my 'cap'. So it helps if, in the morning, I have a clear plan of
campaign before I start. This morning I just wanted the programme for us to
watch later on.

BTW my impression is that there are actually more than two output-facing
servers. Yet here we only talk of akamai and limelight. Do the two name
cluster together more than one actual server system, or is something
missing? ...Or is this something mortal man is not meant to know? 8-]

BTW2 I only tried each method once this morning, although one did have
retries set to twice I think. I'll need to check when  get access to the TV
again. SWMBO is currently watching something using it.
I'll try again when I get a chance to see if I can get the 1280x720
version.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer