Hello everyone, It has been almost a month and a half since my last update here, so I thought I would let everyone know that I am still alive and kickin'.
I have just tracked down and fixed the bug that kept us from being able to load the static DSP patch from TCS211, and thereby brought us one step closer to having working voice calls in our gcc-built libre GSM firmware. Just to recap: between July and now, our gcc-built fw was able to connect to GSM networks and send and receive SMS, but voice calls were totally failing: a call would typically drop before it even finished connecting. We were then running without any DSP patches, whereas all production firmwares known to us always apply some. An experiment showed that if we knock out both static and dynamic DSP patches in our working TCS211/leo2moko reference, it also starts failing on voice calls in a manner similar to our own work-in-progress fw, so it was a reasonable conclusion that at least the static patch is necessary. But when we enabled downloading of this static patch (with actual patch code extracted from TCS211) in our gcc-built gsm-fw, other things started breaking and the DSP was reporting a checksum different from the expected value, indicating that the patch must be getting corrupted somehow. Tracking down the cause of this DSP patch corruption was not trivial at all. At first I thought something was wrong in the actual process of downloading the patch (which comes as a char array) into the DSP, but I couldn't find anything wrong there. Then I decided to examine the structure of this DSP patch: even though it comes as a raw char array, the code that downloads it (leadapi.c, came as real source with our copy of TCS211) understands enough of its structure to figure out where one section ends and another begins, and where in the DSP's RAM address space each section needs to be loaded. Thankfully one of the docs we have for the Calypso chipset hardware shows the memory map of the DSP - just a bird's eye view thereof, but enough to figure out what the download section addresses in the patch correspond to. The DSP "thinks" in terms of 16-bit words, not bytes, and its addresses are word addresses, not byte: an increment of 1 in the DSP address is an advance of 16 bits, not 8. Think of it like the good old PDP-11 (yes, I'm showing my mental age here) where memory sizes were traditionally measured in words instead of bytes. The Calypso DSP has a total of 28 Kwords of RAM (much less RAM than mask ROM: it has 128 Kwords of the latter), and 8 Kwords out of that constitute dual-ported API RAM, accessible to both the DSP and the ARM part of the Calypso (the MCU as TI called it). DSP word addresses from 0 to 0x6FFF map to the RAM (28 Kwords in total), and out of that total RAM address range, DSP word addresses 0x800 through 0x27FF cover the 8 Kword portion that is also visible from the ARM side as the API RAM. On the ARM side this API RAM appears at 0xFFD00000, and 8 Kwords become 16 Kbytes. When I examined the structure of the "base" (statically loaded before any of the dynamic ones) DSP patch from TCS211, I saw that all code download sections except one go into non-API RAM, i.e., into DSP RAM areas which only the DSP can access - this part was as expected. But there is one code section that gets loaded into an address range in the API RAM, and appears to stay there. That section naturally drew my attention: being in a RAM that is accessible to both DSP and ARM cores makes it more susceptible to corruption. Then I wrote a utility that reads and dumps the content of the DSP API RAM in a Calypso GSM device as the regular fw is running, using the ETM memory read command in 16-bit mode. This utility has been added to the FreeCalypso host tools suite as fc-dspapidump. I ran this utility against our "golden" TCS211/leo2moko reference, and indeed the content of those API RAM words was exactly the same as in the patch, confirming my assumption that these API RAM words are reserved for that part of the patch code and not reused for something else. Then I did the same fc-dspapidump from our failing fw, after it downloaded the patch and got the wrong checksum back (0x15C8 instead of the expected 0xD278). Observed result: one word in the API RAM range in question was different. API RAM word 0xEF9 (0x16F9 as addressed by the DSP, 0xFD001DF2 as addressed by the ARM) contains 0x4350 in the working TCS211 case, as loaded from the patch code array, but in the case of our failing fw it contained 0x0000. A quick bit of hex math confirmed that this word must be responsible for the checksum mismatch: 0xD278 + 0x4350 = 0x15C8 with 16-bit word overflow. (The checksum function must be adding all words together as a simple ripple carry sum and then taking either a 1's or a 2's complement.) Then I went over our L1 code, looking to see if I could find any references to the DSP API word in question, and found the culprit. It was a LoCosto-ism that slipped through the cracks. As a reminder, most of the difficulties we are having in our FC project in general stem from the destructive nature of Man: many of TI's sources and docs from the Calypso era apparently have not survived the mainstream society's reclassification of the Calypso as "obsolete junk" followed by the closure of the TI division in question, and we are left with just bits and pieces to work with. A source for a Calypso version of L1 that would correspond closely to the "golden" version we've got in binary object form is one of those things which we lack. Aside from the TSM30 version which has other problems, the only TI L1 source we have comes from their LoCosto program, not Calypso, and this LoCosto L1 source formed the basis for the L1 code in our current FC GSM fw. TI did not follow what I would consider to be good software development practices. If you've got an older chip and working firmware for it, and now you've got a newer chip and you need to make the fw run on the latter, usually the responsible course of action would be to add support for the newer chip without breaking the old one, inserting conditional compilation directives as necessary. Then you would have just one source to maintain, building different binaries for different hw targets, with features that make sense only on some (newer) targets cleanly disabled on those hw targets where they can't work, while target hw-independent improvements automatically benefit the older hw as well. You would regression-test your fw on the older targets to make sure you didn't break them. This is the approach taken by the Linux kernel, for example. But TI failed to do the above with their L1 code. When they ported L1 from Calypso to LoCosto, they only did lip service to keeping it backward-compatible with the Calypso. There are plenty of C preprocessor conditionals on CHIPSET, DSP, ANLG_FAM and RF_FAM (preprocessor-defined magic numbers identifying different hw targets), but they did not regression-test this new L1 code on the Calypso, and in actuality the Calypso support in the LoCosto version of L1 is broken: if one tries to compile that LoCosto L1 source with the preprocessor symbols (CHIPSET, DSP etc) set to their Calypso values, the code doesn't even compile. I had to fix all those compilation failures in the process of integrating that code into our FC gsm-fw. Most of the trouble spots in the LoCosto version L1 take the form of TI having added some LoCosto-ism without taking care to put it under conditional compilation, thereby breaking the Calypso configuration. Most of these were caught and fixed in the process of getting the code to compile, as the offending LoCosto-ism would usually break compilation by referencing something that doesn't exist in the Calypso configuration. But some slipped through, and one of these crawling critters was the cause of our DSP patch corruption. One of the new features added with LoCosto is measurement of the load on the DSP CPU. Apparently it requires the new DSP ROM version in the LoCosto, as in several places the code in question is conditionalized on #if (DSP >= 38). Our Calypso DSP is 36. But there were some spots where code was added for the DSP CPU load measurement feature without being conditionalized thus. I fixed most of these back in the summer of 2014 when I was integrating this L1 into our gsm-fw, as they were breaking compilation. But two spots remained: the definition of the DSP API RAM addresses for this feature, and the writing of one of those words from the ARM side. Yup, you probably guessed it by now: the DSP API words used by the DSP CPU load measurement feature on LoCosto (0xEF0-0xEF9) fall into the range used by our DSP patch from TCS211 on the Calypso. Removing the offending write into the 0xEF9 word from the l1ddsp_end_scenario() function in l1_drive.c (putting the #if (DSP >= 38) conditional where TI neglected to do so) made things quite a bit better. The two most recent commits to the freecalypso-sw Hg tree fix this LoCosto creepage bug and enable downloading of the static DSP patch from TCS211. With these changes applied, we are now considerably closer to working voice calls, although we still aren't quite there yet. With non-broken static DSP patch loading enabled, voice calls now connect reliably, whereas previously connection establishment was failing. However, the downlink audio (heard in the earpiece of the GTA02 with the right ALSA scenario file loaded from the AP side) consists of nothing but noise. The uplink audio transmitted toward the far end of the call seems to be OK, though. I was doing all of my tests in this series on the GTA02 target, connecting to the live GSM network of Operator 310260. The debug trace output I get seems to indicate that the codec selected by the network was AMR. I invite DS to test this latest fw fix against his lab network with the plain FR codec. It is worth noting that our current state of voice calls connecting, uplink audio being apparently good, but total noise on the downlink is exactly the same as what Akib Sayyed reported on the OsmocomBB mailing list almost exactly 2 y ago when he tried enabling AMR in OsmocomBB without applying any DSP patches. If we are to continue on the path of trying to make LoCosto-source-based L1 work on Calypso GSM devices in our current gsm-fw framework, the logical next step would be to try integrating the dynamic DSP patch downloading code - as discussed on this list back in July, our TCS211 "golden reference" (leo2moko) loads some dynamic DSP patches on top of the static one, and we've only got the latter in our current gsm-fw. I should also note that for some reason our "golden reference" TCS211 fw seems much more resilient to the lack of DSP patches than our gcc-built one. When I knocked the dynamic patch downloading out in leo2moko while keeping the static one, not only was it still able to make calls, but even the audio still sounded OK to my not-too-discerning ear. It may have been slightly degraded compared to the fully patched version (some artifacts *may* have been introduced, or maybe not - my ears aren't discerning enough to tell reliably), but it was a far cry from what we are getting with our current gsm-fw with static patch only: nothing but noise heard in the earpiece. And when I knocked the static patch out as well, voice calls would *sometimes* fail the same way they failed in our gcc-built fw w/o DSP patches - but other times they would connect just fine, and the audio sounded fine too! In contrast, I never succeeded in dialing a MO call from sans-patches gcc-built fw, and I only got MT calls to succeed once or twice - and I didn't have the ability to check the audio at that time. Getting the dynamic patch download mechanism into our gsm-fw won't be a trivial "just try it" late-one-evening kind of task. The l1_dyn_dwl code in the LoCosto L1 source takes up 6 C modules with a total of 2819 lines of code. And it won't be as simple as just plopping these 6 C files into our gsm-fw tree and fixing a few compilation hitches: the version in the LoCosto source supports the dynamic patches needed for Calypso+ and LoCosto, but not the ones featured in our TCS211 binary libs version. So once again the binary libs will need to be disassembled, the logic contained therein will need to be reverse- engineered, and then reapplied to the available source. When I am ready to tackle this L1 firmware mess once again (see below), I am wondering if perhaps I should take a different approach with the l1_dyn_dwl code. My previous approach has been to plop bits of code from the LoCosto source into gsm-fw, get them to compile with gcc, and have no way of knowing whether or not I got it right - just because something compiles doesn't mean that it doesn't contain a bad porting artifact, as we learned painfully with the static patch corruption bug. I wonder if it might make more sense to take a different approach: instead of plopping bits of code from LoCosto source into gsm-fw, plop them into a TCS211 tree instead, and try to use that nasty build system (which does work under Wine) to compile them with TI's proprietary compiler. Tackle just one C module at a time (don't try to port the whole fw before having any idea if I got it right or not), use the original TCS211 header files (which we do have), and try to get each individual C module in question to compile (with TI's original compiler) into an exact match of the corresponding blob from the semi-src deliverable we got. Before trying to recreate the full TCS211 L1 in that manner, try doing it just for the 6 l1_dyn_dwl_*.c modules. If we can reconstruct TCS211 source for these 6 modules, then we can try integrating them into our gcc-built gsm-fw, and see what happens. If adding this dynamic patch downloading makes voice calls work beautifully, great. If we continue having seemingly-unsolvable problems with this L1, then try reconstructing the rest of TCS211 L1 in the one C module at a time manner. Then try integrating that reconstructed version of L1 into our gcc-built gsm-fw. Yes, a slow and painful way, but we may not have a better alternative. But I am also unsure if doing the above is what I should do next. You see, because there is only one of me, for as long as I am pecking away at firmware reconstruction and reintegration issues, no progress is being made at all on the hardware subproject of FreeCalypso. I realize that many people here probably want to see libre fw running on Mot C1xx phones (or on the Pirelli DP-L10) more than anything else, but I have a broader vision. The perspective of a developer is different from that of a (potential) user. On the one hand, it is reasonable to expect that when we get some libre fw working on one of Motorola or Pirelli targets, non- developer users will likely be content with using just those phones and nothing else. But the needs of a developer are different. The need to be able to play with TCS211 fw in addition to our own will NOT go away any time soon. And for the purpose of running TCS211, Mot and Pirelli phones will *never* be an acceptable replacement for the GTA02 modem. The only thing that can be a fully satisfactory replacement for Openmoko devices for the purposes of FreeCalypso *development* (as opposed to the needs of non-developer users) is our own hardware, the kind I have been proposing for months now. Right now our project is critically dependent on Openmoko devices for development, and because these devices have fully reached unobtainium status, we have no possibility of attracting any new developers. Once again, Mot C1xx or Pirelli DP-L10 may be good enough for a user, but they can never replace the GTA02 modem for developer purposes. And on top of that, this one and only suitable development platform we have is inconvenient as all hell. Our own proposed FC GSM development board will be far more convenient in every way. Thus I feel a need to spend more time and effort on the hardware subproject of FreeCalypso. The current status with the proposed FCDEV3B board (FreeCalypso development board, triband, based on a verbatim reuse of Openmoko's modem section at the physical layout level) is that I got a PADS ECO file made along with the necessary libraries, allowing a PADS-using and PADS-experienced PCB layout engineer to finish the job of creating the FCDEV3B PCB from the GTA02. However, I have not succeeded in finding anyone who would be willing to do this job for a price I can afford. The only ones who were willing to even take a serious look into it were San Diego PCB (www.sdpcb.com), a fully commercial PCB layout shop charging fully commercial labor rates. They said it would take about 40 hours of work at $95/hour. In other words, about 4 kUSD. And that cost is NOT for physically making the board - it is just for finishing the PCB design in PADS, i.e., pure desk work, nothing physical. Whatever it will cost to actually produce the PCB, that will be an entirely separate cost afterward. Needless to say, neither I nor probably our community as a whole can afford to pay 4 kUSD just for the desk work of massaging an already existing PCB design into our desired shape. Thus we need a different approach. When I first got a hold of the GTA02 PCB design in PADS ASCII format (as opposed to PADS binary), my first thought was to write an automated lossless converter from this PADS ASCII format into some FLOSS PCB format. The problem, however, is that the only FLOSS PCB program I am comfortable with (GNU PCB, more commonly referred to as gEDA/PCB) currently lacks support for blind & buried vias and a few other common PCB layout features which are used by the GTA02 design and all other GSM device PCB designs I've seen. It should be obvious that an automated lossless translation from format A into format B can only be possible if B supports all features of A, or at least all features of A which are actually used by the source file to be converted. Thus before we can use a FLOSS PCB layout program to finish the design of our desired FCDEV3B via translation/import from PADS, we will first need to *develop* a FLOSS PCB layout tool that has enough PADS-like capabilities (blind & buried vias, copper pours, pad stacks, arbitrary drawing layers) in order to be able to import a reuse of a commercial GSM device PCB layout that was done in PADS. Thus I feel that the right course of action for me at this point is to embark on a project of creating an improved version of GNU PCB with the just-listed PADS- like capabilities. Comments and suggestions are welcome. Happy hacking, Mychaela aka Space Falcon _______________________________________________ Community mailing list Community@freecalypso.org https://www.freecalypso.org/mailman/listinfo/community
