- Revision
- 233100
- Author
- [email protected]
- Date
- 2018-06-22 14:36:53 -0700 (Fri, 22 Jun 2018)
Log Message
Incoming G722 doesn't work
https://bugs.webkit.org/show_bug.cgi?id=186307
<rdar://problem/40809745>
Reviewed by Eric Carlson.
Source/WebCore:
WebRTC backends usually does the following:
- Initially call RealtimeIncomingAudioSource with 16KHz data
- Switch to 48KHz when actual data is decoded.
We added a check that was discarding any 16KHz data, but in case of G722, the data remains as 16KHz and is then never read.
The solution is to remove the check that discards 16KHz information.
We then need to fix a bug in AudioTrackPrivateMediaStreamCocoa that was preventing proper handling of change of audio data configuration.
Test: webrtc/audio-peer-connection-g722.html
* platform/mediastream/mac/AudioTrackPrivateMediaStreamCocoa.cpp:
(WebCore::AudioTrackPrivateMediaStreamCocoa::audioSamplesAvailable):
* platform/mediastream/mac/RealtimeIncomingAudioSourceCocoa.cpp:
(WebCore::RealtimeIncomingAudioSourceCocoa::OnData):
LayoutTests:
* webrtc/audio-peer-connection-g722-expected.txt: Added.
* webrtc/audio-peer-connection-g722.html: Added.
* webrtc/routines.js:
Modified Paths
Added Paths
Diff
Modified: trunk/LayoutTests/ChangeLog (233099 => 233100)
--- trunk/LayoutTests/ChangeLog 2018-06-22 21:20:32 UTC (rev 233099)
+++ trunk/LayoutTests/ChangeLog 2018-06-22 21:36:53 UTC (rev 233100)
@@ -1,3 +1,15 @@
+2018-06-22 Youenn Fablet <[email protected]>
+
+ Incoming G722 doesn't work
+ https://bugs.webkit.org/show_bug.cgi?id=186307
+ <rdar://problem/40809745>
+
+ Reviewed by Eric Carlson.
+
+ * webrtc/audio-peer-connection-g722-expected.txt: Added.
+ * webrtc/audio-peer-connection-g722.html: Added.
+ * webrtc/routines.js:
+
2018-06-22 David Fenton <[email protected]>
LayoutTests imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSASSA-PKCS1-v1_5.https.any.html and imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.html are flaky.
Added: trunk/LayoutTests/webrtc/audio-peer-connection-g722-expected.txt (0 => 233100)
--- trunk/LayoutTests/webrtc/audio-peer-connection-g722-expected.txt (rev 0)
+++ trunk/LayoutTests/webrtc/audio-peer-connection-g722-expected.txt 2018-06-22 21:36:53 UTC (rev 233100)
@@ -0,0 +1,3 @@
+
+PASS Basic G722 audio playback through a peer connection
+
Added: trunk/LayoutTests/webrtc/audio-peer-connection-g722.html (0 => 233100)
--- trunk/LayoutTests/webrtc/audio-peer-connection-g722.html (rev 0)
+++ trunk/LayoutTests/webrtc/audio-peer-connection-g722.html 2018-06-22 21:36:53 UTC (rev 233100)
@@ -0,0 +1,68 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <meta charset="utf-8">
+ <title>Testing local audio capture playback causes "playing" event to fire</title>
+ <script src=""
+ <script src=""
+ <script src =""
+ <script>
+var context = new webkitAudioContext();
+var remoteStream;
+
+async function checkForHumBipBop(stream, previousResults, counter)
+{
+ if (!previousResults)
+ previousResults = {
+ heardHum : false,
+ heardBip : false,
+ heardBop : false
+ };
+ if (!counter)
+ counter = 1;
+ else if (++counter > 4)
+ return Promise.resolve(false);
+ results = await analyseAudio(stream, 1000, context);
+ previousResults.heardHum |= results.heardHum;
+ previousResults.heardBip |= results.heardBip;
+ previousResults.heardBop |= results.heardBop;
+ if (previousResults.heardHum && previousResults.heardBip && previousResults.heardBop)
+ return Promise.resolve(true);
+ var results = await checkForHumBipBop(stream, previousResults, counter);
+ return results;
+}
+
+function setCodec(sdp, codec)
+{
+ return sdp.split('\r\n').filter(line => {
+ return line.indexOf('a=fmtp') === -1 && line.indexOf('a=rtcp-fb') === -1 && (line.indexOf('a=rtpmap') === -1 || line.indexOf(codec) !== -1);
+ }).join('\r\n');
+}
+
+promise_test(async (test) => {
+ if (window.testRunner)
+ testRunner.setUserMediaPermission(true);
+
+ const stream = await navigator.mediaDevices.getUserMedia({audio: true});
+ const remoteStream = new Promise((resolve, reject) => {
+ createConnections((firstConnection) => {
+ firstConnection.addTrack(stream.getAudioTracks()[0], stream);
+ }, (secondConnection) => {
+ secondConnection._ontrack_ = (event) => { resolve(event.streams[0]); };
+ }, { observeOffer : (offer) => {
+ offer.sdp = setCodec(offer.sdp, "722");
+ return offer;
+ }
+ });
+ setTimeout(() => reject("Test timed out"), 5000);
+ });
+
+ const results = await checkForHumBipBop(stream);
+ assert_true(results, "heard hum bip bop");
+ context.close();
+}, "Basic G722 audio playback through a peer connection");
+ </script>
+</head>
+<body>
+</body>
+</html>
Modified: trunk/LayoutTests/webrtc/routines.js (233099 => 233100)
--- trunk/LayoutTests/webrtc/routines.js 2018-06-22 21:20:32 UTC (rev 233099)
+++ trunk/LayoutTests/webrtc/routines.js 2018-06-22 21:36:53 UTC (rev 233100)
@@ -31,8 +31,11 @@
function gotDescription1(desc, options)
{
- if (options.observeOffer)
- options.observeOffer(desc);
+ if (options.observeOffer) {
+ const result = options.observeOffer(desc);
+ if (result)
+ desc = result;
+ }
localConnection.setLocalDescription(desc);
remoteConnection.setRemoteDescription(desc).then(() => {
Modified: trunk/Source/WebCore/ChangeLog (233099 => 233100)
--- trunk/Source/WebCore/ChangeLog 2018-06-22 21:20:32 UTC (rev 233099)
+++ trunk/Source/WebCore/ChangeLog 2018-06-22 21:36:53 UTC (rev 233100)
@@ -1,3 +1,25 @@
+2018-06-22 Youenn Fablet <[email protected]>
+
+ Incoming G722 doesn't work
+ https://bugs.webkit.org/show_bug.cgi?id=186307
+ <rdar://problem/40809745>
+
+ Reviewed by Eric Carlson.
+
+ WebRTC backends usually does the following:
+ - Initially call RealtimeIncomingAudioSource with 16KHz data
+ - Switch to 48KHz when actual data is decoded.
+ We added a check that was discarding any 16KHz data, but in case of G722, the data remains as 16KHz and is then never read.
+ The solution is to remove the check that discards 16KHz information.
+ We then need to fix a bug in AudioTrackPrivateMediaStreamCocoa that was preventing proper handling of change of audio data configuration.
+
+ Test: webrtc/audio-peer-connection-g722.html
+
+ * platform/mediastream/mac/AudioTrackPrivateMediaStreamCocoa.cpp:
+ (WebCore::AudioTrackPrivateMediaStreamCocoa::audioSamplesAvailable):
+ * platform/mediastream/mac/RealtimeIncomingAudioSourceCocoa.cpp:
+ (WebCore::RealtimeIncomingAudioSourceCocoa::OnData):
+
2018-06-22 Timothy Hatcher <[email protected]>
Recalc styles every time defaultAppearance changes.
Modified: trunk/Source/WebCore/platform/mediastream/mac/AudioTrackPrivateMediaStreamCocoa.cpp (233099 => 233100)
--- trunk/Source/WebCore/platform/mediastream/mac/AudioTrackPrivateMediaStreamCocoa.cpp 2018-06-22 21:20:32 UTC (rev 233099)
+++ trunk/Source/WebCore/platform/mediastream/mac/AudioTrackPrivateMediaStreamCocoa.cpp 2018-06-22 21:36:53 UTC (rev 233100)
@@ -194,8 +194,7 @@
m_inputDescription = std::make_unique<CAAudioStreamDescription>(inputDescription);
m_outputDescription = std::make_unique<CAAudioStreamDescription>(outputDescription);
- if (!m_dataSource)
- m_dataSource = AudioSampleDataSource::create(description.sampleRate() * 2);
+ m_dataSource = AudioSampleDataSource::create(description.sampleRate() * 2);
if (m_dataSource->setInputFormat(inputDescription) || m_dataSource->setOutputFormat(outputDescription)) {
AudioComponentInstanceDispose(remoteIOUnit);
@@ -202,6 +201,12 @@
return;
}
+ if (m_isPlaying && AudioOutputUnitStart(remoteIOUnit)) {
+ AudioComponentInstanceDispose(remoteIOUnit);
+ m_inputDescription = nullptr;
+ return;
+ }
+
m_dataSource->setVolume(m_volume);
m_remoteIOUnit = remoteIOUnit;
}
Modified: trunk/Source/WebCore/platform/mediastream/mac/RealtimeIncomingAudioSourceCocoa.cpp (233099 => 233100)
--- trunk/Source/WebCore/platform/mediastream/mac/RealtimeIncomingAudioSourceCocoa.cpp 2018-06-22 21:20:32 UTC (rev 233099)
+++ trunk/Source/WebCore/platform/mediastream/mac/RealtimeIncomingAudioSourceCocoa.cpp 2018-06-22 21:36:53 UTC (rev 233100)
@@ -68,15 +68,6 @@
void RealtimeIncomingAudioSourceCocoa::OnData(const void* audioData, int bitsPerSample, int sampleRate, size_t numberOfChannels, size_t numberOfFrames)
{
- // We may receive OnData calls with empty sound data (mono, samples equal to zero and sampleRate equal to 16000) when starting the call.
- // FIXME: For the moment we skip them, we should find a better solution at libwebrtc level to not be called until getting some real data.
- if (sampleRate == 16000 && numberOfChannels == 1)
- return;
-
- ASSERT(bitsPerSample == 16);
- ASSERT(numberOfChannels == 1 || numberOfChannels == 2);
- ASSERT(sampleRate == 48000);
-
CMTime startTime = CMTimeMake(m_numberOfFrames, sampleRate);
auto mediaTime = PAL::toMediaTime(startTime);
m_numberOfFrames += numberOfFrames;
