Bugs item #3558019, was opened at 2012-08-15 12:08
Message generated for change (Comment added) made by mhartzel
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=110706&aid=3558019&group_id=10706
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Mikael Hartzell (mhartzel)
Assigned to: Nobody/Anonymous (nobody)
Summary: Wav header incorrect when 4 GB file size exceeded
Initial Comment:
Bug: Wav file header gets incorrect information when 4 GB file size is exceeded
--------------------------------------------------------------------------------------------------------------------
OS: Ubuntu 12.04
Version: Sox version v14.4.2 built from latest git commit (dated 2012.08.07):
35eebaf602ffd1477da1b604d345fab37aa8d42c
How to reproduce bug
---------------------
- Create two 6 channel 48 kHz, 16 bit audio files, one with duration of 2 hours
4 minutes and the other with 2 hours and 5 minutes.
- The latter will exeed 4 GB barrier and wav header file duration and sample
count information gets mixed up.
- Get file duration from both files with sox --i and ffprobe (is included in
FFmpeg).
- Sox --i will tell file durations are: 02:04:00.00 and 00:00:43.46 though they
should both be 02:04:00.00. Also sample count is from in latter file.
- ffprobe will tell both files are 02:04:00.00, I don't know how it gets the
information correctly since sox --i does not.
- VLC will only play 00:00:43.46 of the latter file.
- The wav header gets mixed up also when converting from flac to wav with sox
if the file size exceeds 4 GB.
Commands to reproduce the bug
-----------------------------
- Create test files by running both of these commands:
sox -V -V -n -b 16 testfile-duration_124min_6_channels.wav trim 0 02:04:00.00
remix 0 0 0 0 0 0
sox -V -V -n -b 16 testfile-duration_125min_6_channels.wav trim 0 02:05:00.00
remix 0 0 0 0 0 0
- Check with ls -l that the file size of both files seems to be correct.
- Use sox --i on both files to get file size.
- Use ffprobe on both files to get file size.
Mikael Hartzell
Text output from my test. Shell commands are underlined:
sox -V -V -n -b 16 testfile-duration_124min_6_channels.wav trim 0 02:04:00.00
remix 0 0 0 0 0 0 ; sox -V -V -n -b 16 testfile-duration_125min_6_channels.wav
trim 0 02:05:00.00 remix 0 0 0 0 0 0
----------------------------------------------------------------------------------------------------------------------
sox: SoX v14.4.2
time: Aug 15 2012 20:58:12
uname: Linux mika-HP-Compaq-nc6320-EV073AV 3.2.0-29-generic-pae #46-Ubuntu
SMP Fri Jul 27 17:25:43 UTC 2012 i686
compiler: gcc 4.6.3
arch: 1248 48 44 L OMP
sox INFO nulfile: sample rate not specified; using 48000
Input File : '' (null)
Channels : 1
Sample Rate : 48000
Precision : 32-bit
sox DBUG wav: Writing Wave file: Microsoft PCM format, 6 channels, 48000
samp/sec
sox DBUG wav: 576000 byte/sec, 12 block align, 16 bits/samp
Output File : 'testfile-duration_124min_6_channels.wav'
Channels : 6
Sample Rate : 48000
Precision : 16-bit
Sample Encoding: 16-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no
Comment : 'Processed by SoX'
sox DBUG remix: 0:
sox DBUG remix: 1:
sox DBUG remix: 2:
sox DBUG remix: 3:
sox DBUG remix: 4:
sox DBUG remix: 5:
sox INFO sox: effects chain: input 48000Hz 1 channels (multi) 32 bits
unknown length
sox INFO sox: effects chain: trim 48000Hz 1 channels (multi) 32 bits
02:04:00.00
sox INFO sox: effects chain: remix 48000Hz 6 channels (multi) 32 bits
02:04:00.00
sox INFO sox: effects chain: dither 48000Hz 6 channels 16 bits
02:04:00.00
sox INFO sox: effects chain: output 48000Hz 6 channels (multi) 16 bits
02:04:00.00
sox DBUG sox: start-up time = 0.001637
sox DBUG input: output buffer still held 2048 samples; dropped.
sox DBUG wav: Finished writing Wave file, 4285440000 data bytes 357120000
samples
sox: SoX v14.4.2
time: Aug 15 2012 20:58:12
uname: Linux mika-HP-Compaq-nc6320-EV073AV 3.2.0-29-generic-pae #46-Ubuntu
SMP Fri Jul 27 17:25:43 UTC 2012 i686
compiler: gcc 4.6.3
arch: 1248 48 44 L OMP
sox INFO nulfile: sample rate not specified; using 48000
Input File : '' (null)
Channels : 1
Sample Rate : 48000
Precision : 32-bit
sox DBUG wav: Writing Wave file: Microsoft PCM format, 6 channels, 48000
samp/sec
sox DBUG wav: 576000 byte/sec, 12 block align, 16 bits/samp
Output File : 'testfile-duration_125min_6_channels.wav'
Channels : 6
Sample Rate : 48000
Precision : 16-bit
Sample Encoding: 16-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no
Comment : 'Processed by SoX'
sox DBUG remix: 0:
sox DBUG remix: 1:
sox DBUG remix: 2:
sox DBUG remix: 3:
sox DBUG remix: 4:
sox DBUG remix: 5:
sox INFO sox: effects chain: input 48000Hz 1 channels (multi) 32 bits
unknown length
sox INFO sox: effects chain: trim 48000Hz 1 channels (multi) 32 bits
02:05:00.00
sox INFO sox: effects chain: remix 48000Hz 6 channels (multi) 32 bits
02:05:00.00
sox INFO sox: effects chain: dither 48000Hz 6 channels 16 bits
02:05:00.00
sox INFO sox: effects chain: output 48000Hz 6 channels (multi) 16 bits
02:05:00.00
sox DBUG sox: start-up time = 0.001569
sox DBUG input: output buffer still held 5632 samples; dropped.
sox DBUG wav: Finished writing Wave file, 25032704 data bytes 360000000 samples
ls -l testfile-duration_12*
----------------------------
-rw-rw-r-- 1 mika mika 4285440080 elo 15 21:10
testfile-duration_124min_6_channels.wav
-rw-rw-r-- 1 mika mika 4320000080 elo 15 21:15
testfile-duration_125min_6_channels.wav
sox --i testfile-duration_12*
-----------------------------
Input File : 'testfile-duration_124min_6_channels.wav'
Channels : 6
Sample Rate : 48000
Precision : 16-bit
Duration : 02:04:00.00 = 357120000 samples ~ 558000 CDDA sectors
File Size : 4.29G
Bit Rate : 4.61M
Sample Encoding: 16-bit Signed Integer PCM
Input File : 'testfile-duration_125min_6_channels.wav'
Channels : 6
Sample Rate : 48000
Precision : 16-bit
Duration : 00:00:43.46 = 2086058 samples ~ 3259.47 CDDA sectors
File Size : 4.32G
Bit Rate : 795M
Sample Encoding: 16-bit Signed Integer PCM
Total Duration of 2 files: 02:04:43.46
ffprobe testfile-duration_124min_6_channels.wav
------------------------------------------------
avprobe version 0.8.3-4:0.8.3-0ubuntu0.12.04.1, Copyright (c) 2007-2012 the
Libav developers
built on Jun 12 2012 16:37:58 with gcc 4.6.3
[wav @ 0x92e4aa0] max_analyze_duration reached
Input #0, wav, from 'testfile-duration_124min_6_channels.wav':
Duration: 02:04:00.00, bitrate: 4608 kb/s
Stream #0.0: Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s
ffprobe testfile-duration_125min_6_channels.wav
------------------------------------------------
avprobe version 0.8.3-4:0.8.3-0ubuntu0.12.04.1, Copyright (c) 2007-2012 the
Libav developers
built on Jun 12 2012 16:37:58 with gcc 4.6.3
[wav @ 0x8555aa0] max_analyze_duration reached
Input #0, wav, from 'testfile-duration_125min_6_channels.wav':
Duration: 02:05:00.00, bitrate: 4608 kb/s
Stream #0.0: Audio: pcm_s16le, 48000 Hz, 5.1, s16, 4608 kb/s
----------------------------------------------------------------------
Comment By: Mikael Hartzell (mhartzel)
Date: 2012-09-12 14:20
Message:
Hi again :)
I was thinking about the case where a user converts a compressed
multichannel audio file to uncompressed wav and I had an idea :)
The easiest and most efficient way to solve this problem would be to not
let sox do any work if it can see that the output file size would go over 4
GB.
When sox starts, it propably has all the information about the input file
it needs and can estimate the uncompressed output file size for example
using sample count, bit depth and channel count. Sox could then output an
error message and refuse to do the conversion if the resulting data can not
be stored in a wav container without breaking the header.
I think this solution might be quite easy to implement and could be used in
all cases where the output file is a 32 bit container.
Mikael Hartzell
----------------------------------------------------------------------
Comment By: Mikael Hartzell (mhartzel)
Date: 2012-09-02 12:28
Message:
Hi :)
Thanks for replying :)
Yes I am aware that the 'old' wav format has a hard limit of 4 GB, but the
way sox handles this limit is in my opinion a little troublesome.
Imagine a scenario where you convert a 8 channel flac file to wav. It is
very possible that the output file size may exceed wav format max limit of
4 GB. Sox does not enforce the limit but instead lets the header become
invalid. Most of the data in the file becomes not playable in standard
players. When sox is used in a script to automate file conversion, this
error goes unnoticed until somebody tries to open and play the file.
May I humbly suggest two possible workarounds :)
1. When writing wav output sox checks for the 4 GB limit, and stops copying
any more data to the ouput file when the limit is reached. Sox should
display an error message informing the user that the output file was
truncated.
2. When writing wav output sox checks for the 4 GB limit and when it is
crossed sox writes a w64 header to the file and continues putting more data
to the output file.
In my opinion it is very important to keep the header in a sane condition
and in sync with the data in the file. Suggestion 1 would be fairly easy to
implement and should prevent creating invalid files without the user
noticing.
I built another open source project that uses your program and I have just
now become to know sox better and I am very impressed what all it can do :)
So Thanks guys for your great work :)
----------------------------------------------------------------------
Comment By: Ulrich Klauer (uklauer)
Date: 2012-08-30 08:23
Message:
Well, there is no way to do this right. RIFF/WAVE only uses 32 bits to
store lengths, so anything >4 GiB won't fit. This is comparable to a paper
form that asks for your age, but only offerss two places for the digits; if
you're 102 years old, you might write "02" or "99", but you can't specify
your true age.
We should perhaps choose the equivalent of "99". But the only way to
completely avoid the problem is for you to switch to a different file
format like W64 or FLAC.
----------------------------------------------------------------------
Comment By: Mikael Hartzell (mhartzel)
Date: 2012-08-15 12:13
Message:
typos creaped in the bug report, corrected lines:
- Sox --i will tell file durations are: 02:04:00.00 and 00:00:43.46 though
they should be 02:04:00.00 and 02:05:00.00. Also sample count is wrong in
latter file.
- ffprobe will tell files are 02:04:00.00 and 02:05:00.00, I don't know how
it gets the information correctly since sox --i does not.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=110706&aid=3558019&group_id=10706
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
SoX-devel mailing list
SoX-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sox-devel
