Re: QtMoko audio state work
Neil Jerram n...@ossau.homelinux.net writes: I'm pretty sure I saw some voice routing problems on a few occasions even with pasuspender. However, since moving back to gta04-gsm-voice-routing, and adding Neil Brown's change to start the 2 capture streams at the same time, I've had a few successful calls and no problems. For the moment, therefore, things are all working for me. I had an unsuccessful call this evening; it was an incoming one. The gsm-voice-routing log for it is below; I plan to analyse this more myself to see if there's a repeating pattern of overruns and underruns, but thought it worth posting in case someone else sees and understands the pattern first. I've also made some minor patches to gta04-gsm-voice-routing and have attached those here. Of course it's possible that it could be one of those that is the problem... Thanks in advance for any insight! Neil gsm-voice-routing started opened p_mod stream opened r_mod stream opened p_ear stream opened r_mic stream voice routing started [2] r_mod: overrun occured: Broken pipe 0 frames available [6] r_mod: overrun occured: Broken pipe 0 frames available [7] p_ear: underrun occured: Broken pipe [7] p_mod: underrun occured: Broken pipe [10] r_mod: overrun occured: Broken pipe 0 frames available [14] r_mod: overrun occured: Broken pipe 0 frames available [15] p_ear: underrun occured: Broken pipe [15] p_mod: underrun occured: Broken pipe [18] r_mod: overrun occured: Broken pipe 0 frames available [22] r_mod: overrun occured: Broken pipe 0 frames available [23] p_ear: underrun occured: Broken pipe [23] p_mod: underrun occured: Broken pipe [26] r_mod: overrun occured: Broken pipe 0 frames available [30] r_mod: overrun occured: Broken pipe 0 frames available [31] p_ear: underrun occured: Broken pipe [31] p_mod: underrun occured: Broken pipe [34] r_mod: overrun occured: Broken pipe 0 frames available [38] r_mod: overrun occured: Broken pipe 0 frames available [39] p_ear: underrun occured: Broken pipe [39] p_mod: underrun occured: Broken pipe [42] r_mod: overrun occured: Broken pipe 0 frames available [46] r_mod: overrun occured: Broken pipe 0 frames available [47] p_ear: underrun occured: Broken pipe [47] p_mod: underrun occured: Broken pipe [50] r_mod: overrun occured: Broken pipe 0 frames available [54] r_mod: overrun occured: Broken pipe 0 frames available [55] p_ear: underrun occured: Broken pipe [55] p_mod: underrun occured: Broken pipe [58] r_mod: overrun occured: Broken pipe 0 frames available [62] r_mod: overrun occured: Broken pipe 0 frames available [63] p_ear: underrun occured: Broken pipe [63] p_mod: underrun occured: Broken pipe [66] r_mod: overrun occured: Broken pipe 0 frames available [70] r_mod: overrun occured: Broken pipe 0 frames available [71] p_ear: underrun occured: Broken pipe [71] p_mod: underrun occured: Broken pipe [74] r_mod: overrun occured: Broken pipe 0 frames available [78] r_mod: overrun occured: Broken pipe 0 frames available [79] p_ear: underrun occured: Broken pipe [79] p_mod: underrun occured: Broken pipe [82] r_mod: overrun occured: Broken pipe 0 frames available [86] r_mod: overrun occured: Broken pipe 0 frames available [87] p_ear: underrun occured: Broken pipe [87] p_mod: underrun occured: Broken pipe [90] r_mod: overrun occured: Broken pipe 0 frames available [94] r_mod: overrun occured: Broken pipe 0 frames available [95] p_ear: underrun occured: Broken pipe [95] p_mod: underrun occured: Broken pipe [98] r_mod: overrun occured: Broken pipe 0 frames available [102] r_mod: overrun occured: Broken pipe 0 frames available [103] p_ear: underrun occured: Broken pipe [103] p_mod: underrun occured: Broken pipe [106] r_mod: overrun occured: Broken pipe 0 frames available [110] r_mod: overrun occured: Broken pipe 0 frames available [111] p_ear: underrun occured: Broken pipe [111] p_mod: underrun occured: Broken pipe [114] r_mod: overrun occured: Broken pipe 0 frames available [118] r_mod: overrun occured: Broken pipe 0 frames available [119] p_ear: underrun occured: Broken pipe [119] p_mod: underrun occured: Broken pipe [122] r_mod: overrun occured: Broken pipe 0 frames available [126] r_mod: overrun occured: Broken pipe 0 frames available [127] p_ear: underrun occured: Broken pipe [127] p_mod: underrun occured: Broken pipe [130] r_mod: overrun occured: Broken pipe 0 frames available [134] r_mod: overrun occured: Broken pipe 0 frames available [135] p_ear: underrun occured: Broken pipe [135] p_mod: underrun occured: Broken pipe [138] r_mod: overrun occured: Broken pipe 0 frames available [142] r_mod: overrun occured: Broken pipe 0 frames available [143] p_ear: underrun occured: Broken pipe [143] p_mod: underrun occured: Broken pipe [146] r_mod: overrun occured: Broken pipe 0 frames available [150] r_mod: overrun occured: Broken pipe 0 frames available [151] p_ear: underrun occured: Broken pipe [151] p_mod: underrun occured: Broken pipe [154] r_mod: overrun occured: Broken pipe 0 frames available
Re: QtMoko audio state work
Radek Polak pson...@seznam.cz writes: I've now reimplemented the GTA04 neoaudioplugin.cpp code so that it moves between audio states by changing those 7 switches, instead of using ALSA state files, and that seems to work well (including for PhoneHeadset, subject to A3 software routing trouble). Nice, imo that's the way how it should work. Alsa states were easiest solution - that's why i used it. Your code could work much better. You can see that change at https://github.com/neiljerram/qtmoko/commit/f8a040e352bf6158baa508b1d7bfdb7e42aa32d0, and if you like try it out on your A4 too. It works nicely for me on A3. For A4 I've added code to set the 'Voice route' control correctly, which should be equivalent to your recent 5mA saving change (84fb56e), but I don't know if I've got it exactly right because that code isn't executed on my A3. Also for A4 it might be necessary to switch the 'Codec Operation Mode' control between 'Option 1 (audio)' and 'Option 2 (voice/audio)', but I haven't written any code for that yet, because - the last email discussion about that didn't seem completely definitive about whether it is really needed - the Phone*.state files in devices/gta04/src/plugins/audiohardware/neo/a4 were inconsistent on this point, i.e. PhoneEarpiece.state had 'Option 2 (voice/audio)' but PhoneSpeaker.state and PhoneHeadset.state had 'Option 1 (audio)' - if we _do_ need to switch 'Codec Operation Mode', we probably need to do that inside pasuspender, so that would make it a bit trickier than the other controls. Therefore it would be nice to know if my change works on A4. One detail, which is nice but slightly worrying, is that I used to always to get a very audible click about 1s before, e.g., hearing the new message arrival sound; and with the reimplementation I no longer get that click. I _think_ the explanation for that was using pasuspender, and that I no longer get it because I no longer need to use pasuspender pasuspender will still have to be used on A4 when switching to earpiece, because you cant switch if some other program has sound card open. Do you mean the switching of 'Codec Operation Mode' here? Is that actually needed for earpiece but not for speaker or headset? If so that would explain what I've described above as inconsistent. - but it's slightly worrying in case that's wrong, and in particular in case it's because I'm leaving some circuit on more than before, and hence drawing more current. (I haven't seen any evidence for drawing extra current.) Maybe it can be measured during suspend. Since having this change in place, I've been seeing the same average suspend currents as before. The simplicity of gta04-gsm-voice-routing is appealing, but I know from previous experience that it sometimes fails completely. For me the problem was that some other program had soundcard open and gta04- gsm-voice-routing couldnt open it. If all programs use pulseadio then it can be solved with pasuspender, but i wish that alsa had the same functionality. Then we could get rid of pulseaudio. Maybe something like this could be achieved using alsa plugins. I'm pretty sure I saw some voice routing problems on a few occasions even with pasuspender. However, since moving back to gta04-gsm-voice-routing, and adding Neil Brown's change to start the 2 capture streams at the same time, I've had a few successful calls and no problems. For the moment, therefore, things are all working for me. Regards, Neil ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko audio state work
On Thursday, January 17, 2013 01:52:57 AM Neil Jerram wrote: (In general, for some reason, it appears that the floating point code in Pulseaudio and its dependencies doesn't run any better on armhf than it does on armel.) Hi Neil, do you have also armhf compiled kernel? I had armel kernel + armhf rootfs and it was even slower then pure armel. Btw i am using armhf for a few weeks now and i think i could make a release - just few apps are missing. Regards Radek ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko audio state work
On Thursday, January 17, 2013 01:52:57 AM Neil Jerram wrote: (As usual, for the sake of avoiding any possible negative marketing, I should point out that the main problems here are A3-specific. I hope, and believe to be the case, that phone calls are already reliable on A4.) Yes, A4 works 100% reliably for me now in last few months, but i have just a few calls... With Neil Brown's helpful input, I've reviewed and improved my understanding of all the ALSA state files and controls. This was prompted specifically by the PhoneHeadset state not working (because it was completely wrong) but more generally it bothers me that we use such an over-general system as ALSA state files when there are really only a handful (in fact, 7 switches and 1 volume control) of independent things that we ever need to change when moving between audio states. Also it has bothered me that we don't have a uniform and persistent way of changing overall volume. I've now reimplemented the GTA04 neoaudioplugin.cpp code so that it moves between audio states by changing those 7 switches, instead of using ALSA state files, and that seems to work well (including for PhoneHeadset, subject to A3 software routing trouble). Nice, imo that's the way how it should work. Alsa states were easiest solution - that's why i used it. Your code could work much better. One detail, which is nice but slightly worrying, is that I used to always to get a very audible click about 1s before, e.g., hearing the new message arrival sound; and with the reimplementation I no longer get that click. I _think_ the explanation for that was using pasuspender, and that I no longer get it because I no longer need to use pasuspender pasuspender will still have to be used on A4 when switching to earpiece, because you cant switch if some other program has sound card open. - but it's slightly worrying in case that's wrong, and in particular in case it's because I'm leaving some circuit on more than before, and hence drawing more current. (I haven't seen any evidence for drawing extra current.) Maybe it can be measured during suspend. The simplicity of gta04-gsm-voice-routing is appealing, but I know from previous experience that it sometimes fails completely. For me the problem was that some other program had soundcard open and gta04- gsm-voice-routing couldnt open it. If all programs use pulseadio then it can be solved with pasuspender, but i wish that alsa had the same functionality. Then we could get rid of pulseaudio. Maybe something like this could be achieved using alsa plugins. Regards Radek ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: [Gta04-owner] QtMoko audio state work
On Thursday, January 17, 2013 01:37:58, NeilBrown wrote: On Thu, 17 Jan 2013 00:52:57 + Neil Jerram n...@ossau.homelinux.net wrote: The upshot of all that is that I'm now inclined to look more again at the other possible solutions, i.e. gta04-gsm-voice-routing (by Radek) and alsaloop (as used in SHR). The simplicity of gta04-gsm-voice-routing is appealing, but I know from previous experience that it sometimes fails completely. The only problems that I've had with gta04-gsm-voice-routing is when the program that plays the alert sound holds on to the audio port for some reason and thus blocks voice-routing from accessing it. I could probably fix that with certainty by a well-placed 'kill' at the start of voice-routing but I want to work out why it is going wrong first (this is a little program of QtMoko tries to cover that by running pasuspender -- gsm-voice-routing instead of just gsm-voice-routing. I think that means that voice routing only starts once pulseaudio has let go of the sound card. mine ... I think it might be confused by getting signals at bad time - I hate programming with signals). :-) I guess that's also why you don't want to fix the problem by adding a kill. alsaloop in comparison has a drastically different and more complex design. I'm wondering if gta04-gsm-voice-routing is unstable _because_ its design is overly simple, and if something more like alsaloop is fundamentally needed - but I haven't yet worked out even how to start analysing that; any ideas would be most welcome. Also, if we did go with alsaloop, I've no idea yet how we might try to add in echo cancellation. alsaloop is 923 lines while gsm-voice-routing is 673 lines. That isn't drastically more complex. The main value-add seems to be sample-rate matching which doesn't seem to be a big problem in practice (if you need it but don't have it you get occasional clicks. I don't get any clicks). There's also the big difference - at least to me - that alsaloop is select-driven, so reads from capture into alsaloop's buffer happen independently of writes from the buffer to playback. What sort of stability problems do/did you experience with gsm-voice-routing? On several occasions, on receipt of a real incoming call, I've just got a kind of distorted quiet growling noise instead of proper audio from the far end. On the other hand, whenever I'm just testing, the audio almost always works. I wonder if the rest of the phone is using more CPU for a real incoming call than when I'm testing, and if that affects how gsm-voice-routing starts up. Well, you've encouraged me to try more with gsm-voice-routing. I think I need to understand more about _how_ it fails, when it does, and I should be able to discover that by adding more logging. Can I just check: is your gsm-voice-routing code the same as in QtMoko? Many thanks, Neil ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko audio state work
On Thursday, January 17, 2013 12:49:04, Radek Polak wrote: On Thursday, January 17, 2013 01:52:57 AM Neil Jerram wrote: (In general, for some reason, it appears that the floating point code in Pulseaudio and its dependencies doesn't run any better on armhf than it does on armel.) Hi Neil, do you have also armhf compiled kernel? I had armel kernel + armhf rootfs and it was even slower then pure armel. Ah, interesting. After I managed to build QtMoko for armhf, I installed it in the experimental wheezy/armhf rootfs that you released a few months ago. So it depends what the kernel in that rootfs was. Btw i am using armhf for a few weeks now and i think i could make a release - just few apps are missing. Nice! I am looking forward to that. Neil ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: [Gta04-owner] QtMoko audio state work
On Thursday, January 17, 2013 02:50:54 PM Neil Jerram wrote: On several occasions, on receipt of a real incoming call, I've just got a kind of distorted quiet growling noise instead of proper audio from the far end. For me the growling took just one 1 second - IMO it was some old recorded sample and after it finished the program was working normally. The biggest problem was for me the volume - once i raised it, the echo cancelation was not working good. Maybe WebRTC has better algorithms then speex? You can also try to play with alsa record/playback buffer sizes. The smaller the buffers are the smaller will be delay between recording and playing. And last thing that can be tried is speex AEC settings. Thats the line: speex_echo_state_init(256, 8192); Basically the gsm-voice-routing should work like this: rec UMTS - SEND to AEC - play on earpiece rec MIC - cancel echo - play on UMTS cancel echo works by substracting rec MIC - rec UMTS. It would be quite interesting to dump the recored sounds and also the AEC'ed sound to files and open them in some program to see how the sounds are shifted in time and to see how the AEC works. IIRC speex has also another cancelling mode besides speex_echo_cancellation() function. Hmmm maybe i could start playing with it again - after one year i can find something new. Regards Radek ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: [Gta04-owner] QtMoko audio state work
On Thu, 17 Jan 2013 14:43:19 +0100 Radek Polak pson...@seznam.cz wrote: The simplicity of gta04-gsm-voice-routing is appealing, but I know from previous experience that it sometimes fails completely. For me the problem was that some other program had soundcard open and gta04- gsm-voice-routing couldnt open it. If all programs use pulseadio then it can be solved with pasuspender, but i wish that alsa had the same functionality. Then we could get rid of pulseaudio. Maybe something like this could be achieved using alsa plugins. The dmix alsa plugin is intended for this purpose. If it is active, then multiple clients can open the device and the sounds they write get mixed together and played. A problem with this is that dmix imposes a fixed (I think) sample size, which implies a fixed latency which is probably more than the latency we want. gsm-voice-routing is quite latency sensitive, both for quality-of-call and for effectiveness of the echo cancelling. NeilBrown signature.asc Description: PGP signature ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: [Gta04-owner] QtMoko audio state work
On Thu, 17 Jan 2013 13:50:54 + Neil Jerram n...@ossau.homelinux.net
wrote:
What sort of stability problems do/did you experience with
gsm-voice-routing?
On several occasions, on receipt of a real incoming call, I've just got a kind
of distorted quiet growling noise
instead of proper audio from the far
end. On the other hand, whenever
I'm just testing, the audio almost
always works. I wonder if the rest of
the phone is using more CPU for a real
incoming call than when I'm testing,
and if that affects how gsm-voice-routing
starts up.
Well, you've encouraged me to try
more with gsm-voice-routing. I think
I need to understand more about _how_
it fails, when it does, and I should be
able to discover that by adding more
logging.
Can I just check: is your gsm-voice-routing
code the same as in QtMoko?
Not exactly :-)
I've just pushed out the code that I'm using to
git://neil.brown.name/gta04-gsm-voice-routing
(master branch).
I thought I'd made more changes than it seems that I have...
Just:
--- a/gsm-voice-routing.c
+++ b/gsm-voice-routing.c
@@ -596,6 +596,11 @@ int main()
open_route_stream_repeated(p0);
open_route_stream_repeated(r0);
+while (route_stream_read(r1))
+ ;
+snd_pcm_start(r0.handle);
+snd_pcm_start(r1.handle);
+
/* Route sound */
while (!terminating) {
NeilBrown
signature.asc
Description: PGP signature
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko audio state work
I wanted to write a bit of an update on my recent GTA04/QtMoko work, and would appreciate people's thoughts. This is still focussed on audio handling, because with my A3 I still haven't reached a point of having fully reliable phone calls. (As usual, for the sake of avoiding any possible negative marketing, I should point out that the main problems here are A3-specific. I hope, and believe to be the case, that phone calls are already reliable on A4.) With Neil Brown's helpful input, I've reviewed and improved my understanding of all the ALSA state files and controls. This was prompted specifically by the PhoneHeadset state not working (because it was completely wrong) but more generally it bothers me that we use such an over-general system as ALSA state files when there are really only a handful (in fact, 7 switches and 1 volume control) of independent things that we ever need to change when moving between audio states. Also it has bothered me that we don't have a uniform and persistent way of changing overall volume. I've now reimplemented the GTA04 neoaudioplugin.cpp code so that it moves between audio states by changing those 7 switches, instead of using ALSA state files, and that seems to work well (including for PhoneHeadset, subject to A3 software routing trouble). One detail, which is nice but slightly worrying, is that I used to always to get a very audible click about 1s before, e.g., hearing the new message arrival sound; and with the reimplementation I no longer get that click. I _think_ the explanation for that was using pasuspender, and that I no longer get it because I no longer need to use pasuspender - but it's slightly worrying in case that's wrong, and in particular in case it's because I'm leaving some circuit on more than before, and hence drawing more current. (I haven't seen any evidence for drawing extra current.) Overall phone volume is controlled by the addition of the three DAC2 Gain controls, and the new state change implementation never touches those, which means that it's now possible for a volume control widget to change those controls independently, and for that setting to persist. I haven't implemented that yet though. Next up is the A3 software audio routing. I announced a while ago that I had that working with pulseaudio's module-loopback instead of Radek's gta04-gsm-voice-routing program, and I thought that pulseaudio would be the way to go. Since then I've tried to add in echo cancellation, and tried running on an wheezy/armhf system instead of squeeze/armel - which is believed to be advantageous because of better floating point performance, but I've hit various problems. - Just plain loopback, with module-loopback, appears to use a lot (~50%) of CPU, even when running at just 8000Hz, without any resampling, and without asking for particularly low latency. I don't recall how much CPU gta04-gsm-voice-routing takes, but I don't think it was that much. - Pulseaudio needs to be run without RT scheduling, in order to avoid being killed (because of tight-looping) during the initial window between when a call is started and when the GSM capture device becomes readable. But running without RT scheduling reduces the quality of media playback. - Pulseaudio loopback exhibits some odd artefacts. During that initial window (of maybe a second or two) it cyclically replays whatever was last in the device's playback buffer. It audibly does this for what is played through the earpiece/speaker/headset; I wonder if it might do it a bit in the other direction too, i.e. to the other end of the call? It also seems to cause occasional short sound repeats _during_ a call. I think one possible cause of this is divergence between the two sound cards' clocks, hence the buffers being used up at different rates, and at some point Pulseaudio has to choose some strategy for filling in some missing data (to avoid an underrun). I've also frequently noticed that a DTMF tone pressed by me seems to have an effect on the other end as though I had pressed the key twice instead of once, and I wonder if that might be related to repeating or echoing in the audio stream going to the other end. - Despite the WebRTC echo cancellation's apparent good reputation, I haven't been able to get either it or Speex to work effectively. Also CPU usage when trying to do loopback with echo cancellation is 80-90%, even on armhf. (In general, for some reason, it appears that the floating point code in Pulseaudio and its dependencies doesn't run any better on armhf than it does on armel.) The upshot of all that is that I'm now inclined to look more again at the other possible solutions, i.e. gta04-gsm-voice-routing (by Radek) and alsaloop (as used in SHR). The simplicity of gta04-gsm-voice-routing is appealing, but I know from previous experience that it sometimes fails completely. alsaloop in comparison has a drastically different and more
Re: QtMoko audio state work
On Sunday, November 25, 2012 02:55:59 PM Neil Jerram wrote: After a day yesterday where the A3 audio didn't work for me in several calls, I decided to take a closer look at the QtMoko audio code. That led to a sequence of small (I think) cleanups, and a reworking of the audio state handling code, and I'd appreciate hearing what you think about that. I've pushed my work-in-progress branch to https://github.com/neiljerram/qtmoko/commits/nj, and the relevant commits are those from https://github.com/neiljerram/qtmoko/commit/0062faf815f5b1ede6624acac08bf3b 8ec7040bd to https://github.com/neiljerram/qtmoko/commit/958212671741685ae05de9e2564ea45 272f985d6 inclusive, excluding https://github.com/neiljerram/qtmoko/commit/18404194e37b85a315a64ae23b3ac97 e7dab3256. In summary, these changes: - remove code that I believe has no effect on the GTA04 - simply so as to make the audio-related code overall easier to understand - simplify and clarify the *AudioState classes and related code in neoaudioplugin.cpp - allow for different ALSA state files for A3 and A4 - rename the state files to match the QtMoko domain (Phone/Media) and profile (Headset/Speaker/Earpiece/Bluetooth) names - make the set of A3 state files more consistent with each other. Apart from the last point, all these changes should just be cleanups and have no practical effect. In particular, on A4 there should be no change at all. On A3 the last point may have a practical effect, if the previous switching of 'AVADC Clock Priority' and 'Voice route' values was sometimes causing a problem. I did a load of test calls this morning and had good audio on all of them. That might just be good luck - or it might indicate that that last point really does make a difference for A3. I guess I'll know better after a bit more experience. Anyway, I'd appreciate hearing if you think this line of work looks worthwhile, or any other thoughts you or others have about it. Yup it looks ok, will test it later but it's alreaady pulled in my git. Thanks! Radek ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko audio state work
Radek Polak pson...@seznam.cz writes: Yup it looks ok, will test it later but it's alreaady pulled in my git. Thanks. Given that, I think it's worth me writing a bit more about where/how my work is going, and there's one bugfix below that you should cherry-pick: please look for While doing that I noticed a bug. Apart from that bugfix I don't want to make any assumptions about what time you have to consider this, so please feel free to leave the rest of this hanging for now. But if you do have time and thoughts I'm sure those would be useful, and in any case it's helpful for me to lay out my thoughts. Firstly, I've just pushed some more commits to https://github.com/neiljerram/qtmoko/commits/nj. These are mostly NOT suitable for your Git master, so please don't pull them, but they may help show what I'm doing. First, my previous set of commits _didn't_ make audio in calls any more reliable. My initial tests were just lucky, and in the days after that I had some calls with no audio, the same as before my cleanups. (In fact that was as expected, because the 'cleanups' didn't really fix anything.) Then I looked for a while at the gta04-gsm-voice-routing code. Empirically, - this code often works - giving clear audio - but sometimes fails catastrophically and gives no audio at all - it seems to fail more often when an incoming call causes the phone to wake up from suspend. My guess is that there is an instability in that code which is more likely to strike when other things on the phone are using CPU - such as just after waking from suspend. Perhaps it could something like this: - CPU load causes gta04-gsm-voice-routing to be slow to read the capture devices, so they report overruns - that may cause the code to loop round and try reading those devices again - which now blocks until time has passed to allow more data to be there - when the code eventually gets to the playback devices, too much time has passed, so they report underrun and don't actually play any data - etc. Then I looked at the alsaloop code and my first impression there was how much more complex it is... Putting everything together, my feeling is that this (loopback) is trickier than it looks to get right, and that gta04-gsm-voice-routing sometimes fails because it doesn't cover all the possible variations. So then I looked at pulseaudio, found Neil Brown's old suggestion, upgraded to the testing version, and eventually stumbled across the resampler method change that makes that work (per other email). I've now integrated that in my code: https://github.com/neiljerram/qtmoko/commit/9f78985e8119961cc520e4d050c88bf1e589b879 I then had to make another change to /etc/pulse/daemon.conf: -; realtime-scheduling = yes + realtime-scheduling = no to avoid pulseaudio being killed by the kernel just after starting the loopback. I guess this is the same basic problem as SHR had with alsaloop here: http://lists.goldelico.com/pipermail/gta04-owner/2012-May/002383.html and ideally the fix would be in pulseaudio to initially spin more slowly. But the realtime-scheduling change works too and doesn't impact general media playback too badly (for my taste). But after that, the integrated pulseaudio in-call audio routing seems to work. Of course I'll be more confident about it if I can have a week of it working every time... While doing that I noticed a bug in my previous Rework ALSA state / QAudioState handling commit: it will call gsmVoiceStart() and gsmVoiceStop() even for A4 devices. That's fixed by https://github.com/neiljerram/qtmoko/commit/a362c431531d6b75fcb1894c60e021588ea50d44 so that's one commit that you _should_ cherry-pick. Next what I'd like to do is to make everything louder! There are 3 things that aren't loud enough at the moment: a) the ringtone b) the in-call audio that I hear from the other person c) the in-call audio that the other person hears from me. (b) and (c) depend on good echo cancellation, and I'm hoping pulseaudio's module-echo-cancel will do that for me. I think I can test that, without needing lots of phone calls, simply by playing something (from file) through the earpiece or speaker and simultaneously recording from the microphone. If that works well, we can then just increase all the volumes in the state files. (BTW I think that the Speex echo cancellation in gta04-gsm-voice-routing was less effective because of the playback buffer being 4 times the frame size. This means that when the code sends frame X to ALSA for playback, frame X doesn't actually start playing until 3 cycles later. Therefore we can't immediately use frame X to cancel echo in the microphone sound that we're capturing _now_. I wonder if there was a time when the code had buffer size = frame size, and the buffer size was later increased?) Finally, if all of the above works, we can look at whether it all _still_ works with the squeeze version of pulseaudio. Hopefully eventually the software audio
Re: QtMoko audio state work
On Saturday, December 01, 2012 06:15:04 PM Neil Jerram wrote: Given that, I think it's worth me writing a bit more about where/how my work is going, and there's one bugfix below that you should cherry-pick: please look for While doing that I noticed a bug. Apart from that bugfix I don't want to make any assumptions about what time you have to consider this, so please feel free to leave the rest of this hanging for now. But if you do have time and thoughts I'm sure those would be useful, and in any case it's helpful for me to lay out my thoughts. Hmm i plan some relaxing now ;-) But after that, the integrated pulseaudio in-call audio routing seems to work. Of course I'll be more confident about it if I can have a week of it working every time... Very nice. While doing that I noticed a bug in my previous Rework ALSA state / QAudioState handling commit: it will call gsmVoiceStart() and gsmVoiceStop() even for A4 devices. That's fixed by https://github.com/neiljerram/qtmoko/commit/a362c431531d6b75fcb1894c60e0215 88ea50d44 so that's one commit that you _should_ cherry-pick. done Next what I'd like to do is to make everything louder! yup, that would be nice There are 3 things that aren't loud enough at the moment: a) the ringtone b) the in-call audio that I hear from the other person c) the in-call audio that the other person hears from me. (b) and (c) depend on good echo cancellation, and I'm hoping pulseaudio's module-echo-cancel will do that for me. I think I can test that, without needing lots of phone calls, simply by playing something (from file) through the earpiece or speaker and simultaneously recording from the microphone. If that works well, we can then just increase all the volumes in the state files. (BTW I think that the Speex echo cancellation in gta04-gsm-voice-routing was less effective because of the playback buffer being 4 times the frame size. This means that when the code sends frame X to ALSA for playback, frame X doesn't actually start playing until 3 cycles later. Therefore we can't immediately use frame X to cancel echo in the microphone sound that we're capturing _now_. I wonder if there was a time when the code had buffer size = frame size, and the buffer size was later increased?) Hmm it was long time ago. IIRC with small buffer size it was not working - but i cant recall the details. :( Finally, if all of the above works, we can look at whether it all _still_ works with the squeeze version of pulseaudio. Hopefully eventually the software audio routing can be good enough for A3 audio quality to be on a par with A4. Yes that would be nice. And even A4 could benefit this - e.g. for recording phone calls. Regards Radek ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko audio state work
On Sunday, November 25, 2012 02:55:59 PM Neil Jerram wrote: Hi Radek, After a day yesterday where the A3 audio didn't work for me in several calls, I decided to take a closer look at the QtMoko audio code. That led to a sequence of small (I think) cleanups, and a reworking of the audio state handling code, and I'd appreciate hearing what you think about that. I've pushed my work-in-progress branch to https://github.com/neiljerram/qtmoko/commits/nj, and the relevant commits are those from https://github.com/neiljerram/qtmoko/commit/0062faf815f5b1ede6624acac08bf3b 8ec7040bd to https://github.com/neiljerram/qtmoko/commit/958212671741685ae05de9e2564ea45 272f985d6 inclusive, excluding https://github.com/neiljerram/qtmoko/commit/18404194e37b85a315a64ae23b3ac97 e7dab3256. Hi Neil, thanks for the patches, i'll take a look at them in a few days. I am going to have a busy weeks at paid work probably until the Christmas so i have to slow down on QtMoko side a bit... Regards Radek ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko audio state work
Radek Polak pson...@seznam.cz writes: Hi Neil, thanks for the patches, i'll take a look at them in a few days. I am going to have a busy weeks at paid work probably until the Christmas so i have to slow down on QtMoko side a bit... No problem, and thanks. I hope that other work goes well! Regards, Neil ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
QtMoko audio state work
Hi Radek, After a day yesterday where the A3 audio didn't work for me in several calls, I decided to take a closer look at the QtMoko audio code. That led to a sequence of small (I think) cleanups, and a reworking of the audio state handling code, and I'd appreciate hearing what you think about that. I've pushed my work-in-progress branch to https://github.com/neiljerram/qtmoko/commits/nj, and the relevant commits are those from https://github.com/neiljerram/qtmoko/commit/0062faf815f5b1ede6624acac08bf3b8ec7040bd to https://github.com/neiljerram/qtmoko/commit/958212671741685ae05de9e2564ea45272f985d6 inclusive, excluding https://github.com/neiljerram/qtmoko/commit/18404194e37b85a315a64ae23b3ac97e7dab3256. In summary, these changes: - remove code that I believe has no effect on the GTA04 - simply so as to make the audio-related code overall easier to understand - simplify and clarify the *AudioState classes and related code in neoaudioplugin.cpp - allow for different ALSA state files for A3 and A4 - rename the state files to match the QtMoko domain (Phone/Media) and profile (Headset/Speaker/Earpiece/Bluetooth) names - make the set of A3 state files more consistent with each other. Apart from the last point, all these changes should just be cleanups and have no practical effect. In particular, on A4 there should be no change at all. On A3 the last point may have a practical effect, if the previous switching of 'AVADC Clock Priority' and 'Voice route' values was sometimes causing a problem. I did a load of test calls this morning and had good audio on all of them. That might just be good luck - or it might indicate that that last point really does make a difference for A3. I guess I'll know better after a bit more experience. Anyway, I'd appreciate hearing if you think this line of work looks worthwhile, or any other thoughts you or others have about it. Regards, Neil ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
