Re: 2.97 failing to produce files

[Timothy]

On Tuesday, December 13, 2016 6:17 PM, "Vangelis forthnet" 
 wrote:
> GiP 2.96 (used by OP prior to updating to 2.97)
> does not pass -stats to FFmpeg 0.8.18,
> so - as you point - remuxing works.
> 
Ah. I did not listen all that closely to the first line of both logs. I just 
assumed both read 2.97, and was a bit baffled at why one had -stats and the 
other one didn't. That teaches me to assume... I was more focused on the FFMpeg 
portion of the logs, as that was where the issue originated. Either way, looks 
like the OP's issues are all solved now I think.

Timothy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Vangelis forthnet]

On Tue Dec 13 22:33:13 GMT 2016, Timothy wrote: 


Actually, it can, if you don't have the -stats option there
(snip)
It was just the -stats option that was the issue


Hi Timothy

I think it's apparent in the portion of my post you quoted 
that I was refering to the combination of GiP 2.97 + FFmpeg 0.8.18, 
which is the one that does not work; 
GiP 2.96 (used by OP prior to updating to 2.97) 
does not pass -stats to FFmpeg 0.8.18, 
so - as you point - remuxing works. 

BTW, all credit goes to you for initially spoting 
the offending switch in the FFmpeg remuxing command.


Best regards, 
Vangelis.


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Timothy]

On Tuesday, December 13, 2016 3:49 PM, "Vangelis forthnet" 
 wrote:
> The original log on GiP 2.97 with obsolete FFmpeg 0.8.18 is pretty
> clear; http://sprunge.us/XYbJ
> FFmpeg can't remux (repackage) downloaded FLV files into .M4A ones.
> (ffmpeg version 0.8.18-6:0.8.18-0+deb7u1)
> 
Actually, it can, if you don't have the -stats option there, evidenced by the 
log at: http://sprunge.us/aJUh
It was just the -stats option that was the issue for iPlayer radio files.

Timothy

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Charles Johnson]

On 13/12/16 09:19, Jim web wrote:

Maybe the building omitted some codecs
because they don't meet some open/free requirements or similar and the
process wasn't told to include them anyway.


That's a good thought. But the reality seems (i haven't combed it 
thoroughly) to be that the Debian build has actually more codec than the 
git build:


diff ffm-git.log ffm.log

(those files being produced by ffmpeg -codecs) The result is here:

http://sprunge.us/TfTH


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Jim web]

In article <42FB9FA5AB734668B7B82B794E7EB888@vasonote>, Vangelis
forthnet
 wrote:

> Now, this is a second issue, unrelated to GiP itself, in that your
> FFmpeg 3.2.2 build  (on Debian Jessie) appears unable to transcode a
> HE-AACv1 m4a file (produced by GiP) to MP3.

I only occasionally built myself a 'new' version of ffmpeg, and my memory
is unreliable. I also don't use Debian directly, but a downstream
derivative distro. However I do git the ffmpeg files from their source, not
my distro repositories.

I'm wondering if the problem here is due to the choice of switches to tell
the building to include all codecs? Maybe the building omitted some codecs
because they don't meet some open/free requirements or similar and the
process wasn't told to include them anyway.

IIRC you may have to configure the files to tell the make process to
include some 'proprietary' codec handling into the final ffmpeg executable.

Alternatively, I've forgotten earlier emails on this, but maybe a version
from the Debian repos omits the relevant codecs for open/free reasons?

Apologies if this is all irrelvant and I've misunderstood. But I've written
it in case it helps.

Someone with a better understanding (and a better memory than myself) may
be able to say more. I'd need to search though my notebooks to find the
instructions I wrote to remind me how to generate a new ffmpeg family to
check.

Jim

-- 
Electronics  http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio  http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc  http://www.audiomisc.co.uk/index.html


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Vangelis forthnet]

On Sun Dec 11 20:41:39 GMT 2016, Charles Johnson wrote:


that's done separately in a script and is just

ffmpeg -i "${f}" $(basename "${f}").mp3


So, to recap,  your original issue (failure to remux FLV audio files
into MP4 container, after upgrading to GiP 2.97) was caused
by the inability of your obsolete FFmpeg v0.8.18 to handle code
introduced in 2.97 - upgrading to FFmpeg 3.2.2 rectified that issue.

Now, this is a second issue, unrelated to GiP itself,
in that your FFmpeg 3.2.2 build  (on Debian Jessie)
appears unable to transcode a HE-AACv1 m4a file
(produced by GiP) to MP3.

If this inability is global (for all flashaaclow files),
the best course of action would be to report it to
some place relevant, either to the Debian Jessie
team or the FFmpeg mailing list
https://lists.ffmpeg.org/mailman/listinfo/ffmpeg-user/

As stated previously, I can't get my hands on a
pre-compiled FFmpeg-3.2.2-win32 binary (that is
also Windows NT6.0 compatible) to test - the
most recent one I got was compiled from code
authored on Dec 4th, 2016.

I created the following batch file

FOR %%N in (*.m4a) DO ffmpeg -i "%%N" -vn "%%~nN.mp3"
pause

and tested various "aaclow" audio files
downloaded by GiP, I was unable to
replicate your bug with that FFmpeg version...

On Mon Dec 12 13:56:02 GMT 2016, Charles Johnson wrote:


I backed up ffmpeg 3.2.2, symlinked the latest build
from here https://www.johnvansickle.com/ffmpeg/
with /usr/bin/ffmpeg


Apologies for being thick, but do you actually
mean the latest git build (gedb4f5d built on 20161211)
or latest 3.2.2 build offered by johnvansickle?
(so as to determine whether the Debian Jessie
flavour of 3.2.2 was at some fault or the 3.2.2
code in general...)


Oh and i got atomicparsley but probably don't need it


AP is always welcome here, since it adds a full
metadata tag + Windows Explorer thumbnail :-)

On Mon Dec 12 10:14:30 GMT 2016, Charles Johnson wrote:

get_iplayer --mode=flashaaclow1 --start=0 --stop=0 --pid 
b084tjt3 --force --verbose --overwrite 2>&1 | tee b084tjt3.log


Two comments:


--mode=flashaaclow1


GiP will, by default, use the first CDN,
so appending "1" to the mode is redundant;
plus, if CDN 1 fails for whatever reason,
no fallback to CDN 2 will be attempted...


--start=0 --stop=0


Though I'm not seeing evidence of these being used
in the linked log (perhaps they negate each other),
why include them in the command to begin with?

From the docs:


--start   Recording/streaming start offset
--stop    Recording/streaming stop offset


Thanks for the help


You are welcome :-)

Vangelis. 



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Vangelis forthnet]

On Sun Dec 11 20:55:57 GMT 2016, Charles Johnson wrote: 


... Have you tried the --ffmpeg-obsolete switch I suggested
with your 0.8.18 version, prior to updating to 3.2.2?


No i didn't do that


...Pity; had you tried and it worked, it would have been valuable info 
for a subset of GiP 2.97 users who are, for one reason or the other, 
stuck at that obsolete FFmpeg v0.8.x - more so if they are not in a 
position to upgrade it (or simply are not willing to...).


Additionally, it would alert the GiP maintainer to update 
the documentation to include v0.8 to the FFmpeg versions 
to which that switch applies (currently it states: 

--ffmpeg-obsolete
Indicates you are using an obsolete version of ffmpeg (<0.7)

)

Regards

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Charles Johnson]

On 11/12/16 20:33, Vangelis forthnet wrote:

It's during the transcoding from HE-AACv1 => MP3
that your copy of FFmpeg barfs...
Perhaps a regression introduced in 3.2.2? 


I think that must be it. I backed up ffmpeg 3.2.2, symlinked the latest 
build from here https://www.johnvansickle.com/ffmpeg/ with 
/usr/bin/ffmpeg, symlinked get_iplayer to 2.97 from git and all is 
golden. Oh and i got atomicparsley but probably don't need it


Why things were going pear-shaped /before/ i upgraded ffmpeg to 3.2.2, i 
don't know and don't any longer care ;)


Thanks for the help



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Charles Johnson]

On 11/12/16 20:33, Vangelis forthnet wrote:

... Have you tried the --ffmpeg-obsolete switch I suggested
with your 0.8.18 version, prior to updating to 3.2.2?


No i didn't do that, since i thought it a good idea to get the latest 
ffmpeg anyway



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Charles Johnson]

On 11/12/16 20:33, Vangelis forthnet wrote:
So please, you need to provide the transcoding ffmpeg command used; 


Actually that's done separately in a script and is just

 ffmpeg -i "${f}" $(basename "${f}").mp3


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Vangelis forthnet]

On Sun Dec 11 19:16:18 GMT 2016, Charles Johnson wrote:


This one is errorful. It uses the latest version (3.2.2) of ffmpeg


... Have you tried the --ffmpeg-obsolete switch I suggested
with your 0.8.18 version, prior to updating to 3.2.2?

As Alan advised, please provide full GiP command used...
I am assuming

get_iplayer --type=radio --pid=b082wtgd --modes=flashaaclow

from the log...


INFO: Mode list: flashaaclow1
(snip)
606.216 kB / 96.90 sec (99.9%)
DEBUG: RTMP_Read returned: 0
Download complete
DEBUG: Closing connection.


= RTMPdump successfully downloaded
the FLV file in its entirety

INFO: Converted file: 
/tmp/Bells_on_Sunday_-_All_Saints_Church_Worcester_b082wtgd_original.partial.m4a


= your updated FFmpeg 3.2.2 version was able to
successfully losslessly remux the "*.partial.m4a.flv" file
to the MP4 container, producing file "*.partial.m4a"

INFO: Recorded file: 
/tmp/Bells_on_Sunday_-_All_Saints_Church_Worcester_b082wtgd_original.m4a

DEBUG: Record using flashaaclow1 mode return code: '0'


Thus, the offending issue reflected in your first post
has been rectified up to this point!
Indeed, had you issued

get_iplayer --type=radio --pid=b082wtgd --modes=flashaaclow

you'd have gotten yourself an .m4a end file.

 INFO: Downloaded Thumbnail to 
'/tmp/Bells_on_Sunday_-_All_Saints_Church_Worcester_b082wtgd_original.jpg'

WARNING: Cannot tag M4A file


It looks as though AtomicParsley is missing
from your system, so MP4 tagging fails.

I am puzzled with the rest of your log;
FFmpeg 3.2.2 tries to transcode the downloaded
.m4a file to .mp3; but this isn't being done
through the --aactomp3 switch in GiP,
because in that case GiP creates an intermediate
"*.partial.mp3.flv" file and transcodes the
downloaded FLV file to an MP3 one,
without any interim M4A one (unless
my aging memory betrays me :-( )

So please, you need to provide the transcoding ffmpeg command used;

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 
'./Bells_on_Sunday_-_All_Saints_Church_Worcester_b082wtgd_original.m4a':
Output #0, mp3, to 
'Bells_on_Sunday_-_All_Saints_Church_Worcester_b082wtgd_original.m4a.mp3':

(snip)
Stream mapping:
Stream #0:0 -> #0:0 (aac (native) -> mp3 (libmp3lame))


It's during the transcoding from HE-AACv1 => MP3
that your copy of FFmpeg barfs...
Perhaps a regression introduced in 3.2.2?
I cannot get my hands on a FFmpeg 3.2.2 win32 binary
that would run on Vista, so can't check myself.

See if

get_iplayer --type=radio --pid=b082wtgd --modes=flashaaclow --force --mp3

succeeds

Other things to try:
1. --radiomode=flashaacstd
(it's AAC LC, so might be compatible with FFmpeg 3.2.2)
2. Downgrade to FFmpeg 3.2.1 or even 3.0
(tested to work OK on Windows,
unknown how it fares in your setup).

One last thing: Is there a specific reason
you prefer the flashmodes to the default
dash and/or hls ones?

I'll be leaving home in a few, so I'll be
unavailable to follow this tonight -
others, feel free to jump in!

Vangelis. 



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Charles Johnson]

On 11/12/16 14:30, Timothy wrote:

You are definitely using an outdated FFMpeg - version 0.8.18. I'm not sure why there's an issue, except that 
on one log the "-stats" switch was given, which your FFMpeg choked on by saying that there was no 
input file given. On the log without the "-stats" switch, everything went swimmingly. I don't know 
the get_iplayer code to make sure that the "-stats" option isn't passed on to FFmpeg.

I suggest you update to the latest FFMpeg first, which should solve all your 
problems.

Timothy


This one is errorful. It uses the latest version (3.2.2) of ffmpeg

Perhaps in this case it's the BBC's fault?

http://sprunge.us/NOaJ


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Vangelis forthnet]

On Sun Dec 11 17:29 GMT 2016, Charles Johnson cehjohn...@googlemail.com 
wrote:



Actually the log files (afaicr) mentioned avconv.
I do have libav installed AND ffmpeg.
tbh i've got thoroughly confused over
the ffmpeg/avconv dichotomy
but perhaps i should try to get
the latest libav for my distro (Debian Jessie)?


Charles, please post back to the whole list
rather than myself personally - others can follow
the conversation and there was nothing private
(I sense) intended in your message to me... :-)

As for what you say, sorry, I'm out of my depths here;
hopefully, other Debian Jessie users will instruct you
how to proceed further, but I think if you
could get latest FFmpeg installed, it'd be best!

Vangelis 



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Vangelis forthnet]

On Sun Dec 11 17:21:58 GMT 2016, I wrote:


Your very old ffmpeg version has been probably affected
by the following GiP commit:

https://github.com/get-iplayer/get_iplayer/commit/9183610030e9e85ff8d2711108feb1c9d6333f97

It appears the combination of  "-loglevel -stats" is
what broke your 0.8.18 FFmpeg version


... Trying to interpret the code in the commit,
it looks as though you need to pass

--ffmpeg-obsolete

in your download command; but I'm not sure
it'll work, since on GiP's help it states:


--ffmpeg-obsolete
Indicates you are using an obsolete version of ffmpeg (<0.7)
that does not support the -loglevel or -stats options,
so  --quiet, --verbose and --debug will not be applied to ffmpeg.


Please report back whether that switch
works for you, so that GiP's help file
could be modified; from your 2.96 log
we know that FFmpeg 0.8.x supports
at least -loglevel switch (but not -stats).

Vangelis.


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Vangelis forthnet]

On Sun Dec 11 09:03:24 GMT 2016, Alan Milewczyk wrote:


it's nearly 24 hours since you posted your query and no replies


... Anyone remembers my last month's post?

http://lists.infradead.org/pipermail/get_iplayer/2016-November/009922.html


> my knowledge of get_iplayer is incomplete :-( ;

not only because I am running it (since Dec 2010) on
one specific platform only (=> I mean Windows here)
(snip)
For this list not to die,
more members should actively participate
into offering help, not just a select few...


With regards to Charles' issue:


as because of attachments, this mail was queued


Not only does the list require plain text format
in the body of the e-mail, the type and size
of allowed attachments is limited.
In the past we had malware issues in zip
attachments, so probably (haven't tested
recently) zip is blocked...
And .txt attachments are limited by size,
off the top of my head I think it's < 100KB
Anyhow, you did the best thing; verbose/debug
logs should be posted on "pastebin" type sites :-)

We'll assume the original GiP 2.97 command is

get_iplayer --type=radio --pid=b084bp81 --modes=flashaaclow

(or some other to the same effect)


why 2.97 consistently produces empty .flv (or m4a) files for me


No, they are most certainly not empty;
according to your log, rtmpdump succeeds
in downloading your file in its entirety
(radiomode=flashaaclow1)

DEBUG: Got Play.Complete or Play.Stop from server. Assuming stream is 
complete


10416.378 kB / 1666.90 sec (99.9%)


It's just the lossless remuxing to the MP4 (.m4a) container that fails:

WARNING: Conversion failed - retaining 
/tmp/Thinking_Allowed_-_Success_and_Luck_-_Cosmopolitanism_and_Private_Education_b084bp81_original.partial.m4a.flv


Have you tried to play the .flv audio file?


(successful 2.96 log and bad 2.97 one linked instead)


There has been very little change codewise between 2.96->2.97:

https://github.com/get-iplayer/get_iplayer/compare/v2.96...v2.97

My gut feeling is Timothy is right; you are running a very old
ffmpeg version for your distro (ffmpeg already on v3.2.2).
Your very old ffmpeg version has been probably affected by
the following GiP commit:

https://github.com/get-iplayer/get_iplayer/commit/9183610030e9e85ff8d2711108feb1c9d6333f97

It appears the combination of  "-loglevel -stats" is that broke
your 0.8.18 FFmpeg version - BTW, with that old version
you won't be able to properly mux hvf tvmodes into MP4;
FFmpeg >=2.5 is required for that...
Follow Timothy's suggestion:

I suggest you update to the latest FFMpeg first, which should solve all 
your problems.


Regards,
Vangelis. 



___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Timothy]

You are definitely using an outdated FFMpeg - version 0.8.18. I'm not sure why 
there's an issue, except that on one log the "-stats" switch was given, which 
your FFMpeg choked on by saying that there was no input file given. On the log 
without the "-stats" switch, everything went swimmingly. I don't know the 
get_iplayer code to make sure that the "-stats" option isn't passed on to 
FFmpeg.

I suggest you update to the latest FFMpeg first, which should solve all your 
problems.

Timothy

On Friday, December 9, 2016 5:52 AM, "Charles Johnson" 
 wrote:
> I don't know enough about the internals to be able to understand why
> 2.97 consistently produces empty .flv (or m4a) files for me. Have sent
> this mail again as because of attachments, this mail was queued
> 
> (successful 2.96 log and bad 2.97 one linked instead)
> 
> The good
> http://sprunge.us/aJUh
> The bad
> http://sprunge.us/XYbJ
> 
> TIA
> 
> 
> ___
> get_iplayer mailing list
> get_iplayer@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer

Re: 2.97 failing to produce files

[Alan Milewczyk]

Hmm, it's nearly 24 hours since you posted your query and no replies, so 
I'll bite.


I'm sorry, I couldn't make head nor tail of the attachments, but my 
immediate question is (assuming you used the command line) "what was the 
exact command issued to get_iplayer?


Regards

Alan

On 09-Dec-16 18:52, Charles Johnson wrote:
I don't know enough about the internals to be able to understand why 
2.97 consistently produces empty .flv (or m4a) files for me. Have sent 
this mail again as because of attachments, this mail was queued


(successful 2.96 log and bad 2.97 one linked instead)

The good
http://sprunge.us/aJUh
The bad
http://sprunge.us/XYbJ

TIA


___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer




___
get_iplayer mailing list
get_iplayer@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/get_iplayer