/gsm/com/rfcap: tri-band GSM modem believes itself to be quad-band

2014-01-15 Thread Michael Spacefalcon
Hello Om community,

In the course of hacking TI GSM firmwares, I have come across something
that some of you may find interesting, or might even have some insight
into.

We all know that our good familiar Neo Freerunner (GTA02) was made in
two versions: one with 900/1800/1900 MHz bands, the other with
850/1800/1900 MHz instead.  The hardware difference is one RF SAW
filter part populated differently on the same PCB; see this picture:

http://wiki.openmoko.org/wiki/File:Gta02a6_comms_chips_under_shield.JPG

The SAW filters for the GSM downlink Rx path are the 3 little buggers
near the upper right corner, immediately adjacent to the shiny metallic
component which is the antenna switch.  There is one Rx SAW filter for
each of the 3 supported bands: one for 1800 MHz (both GTA02 versions),
one for 1900 MHz (ditto), and one populated for either 850 or 900 MHz.
(*I think* the topmost one out of the 3 in that picture is the one
responsible for the 850 vs. 900 MHz difference, but please double-check
that before attempting any surgery on your Neo!)

Well, here is the part which will surely surprise at least some of you:
the standard firmware for the GTA01/02 modem (which is the same for
all versions, both GTA01 and GTA02) does not actually know which 3
frequency bands are supported by the device it runs on!  And no, it
does not auto-detect either: there is no way (short of ESP) for any
firmware running on the Calypso/Iota/Rita chipset to divine what kind
of SAW filter sits between that chipset and the antenna.

Instead, as strange as it may sound, the modem (at least when running
the standard mokoN firmware, see below) believes itself to be quad-
band!

In TI's universe, the standard way to teach a GSM device (phone or
modem) which GSM frequency bands it supports is *not* to hard-code
that knowledge in the firmware at compile time; instead this property
is stored in a configuration file named /gsm/com/rfcap in the GSM
device file system.  Yes, TI-based GSM devices all use a flash file
system with a very UNIX-like look and feel, including UNIX-style
pathnames; see my write-up:

https://bitbucket.org/falconian/freecalypso-sw/src/1852900ce9ea4ac52d4648f7d9ca46897eb3640b/doc/TIFFS?at=default

Like most files in TI's GSM FFS, /gsm/com/rfcap is a binary file, not
ASCII.  It is a file of exactly 16 bytes, and although I haven't found
a formal document describing its format in plain English, we can study
the code that reads this file and acts upon its content:

http://scottn.us/downloads/peek/TCS3.2_N5.24_M18_V1.11_M23BTH_PSL1_src/g23m-gsm/rr/rr_csf.c

The 16-byte file is being read into a variable of type EF_RFCAP, which
is defined here:

http://scottn.us/downloads/peek/TCS3.2_N5.24_M18_V1.11_M23BTH_PSL1_src/g23m/condat/com/include/pcm.h

Lines 442 through 460 (inclusive) give the structure definition, which
is followed by the definitions for the bit fields in each byte.

And here is what this /gsm/com/rfcap file contains on a standard GTA02
modem, as revealed by a TIFFS parsing tool such as the mpffs-tools-r1
package I released last summer:

00 1F 41 14 00 00 00 00  50 00 00 A5 05 00 C0 00

Decoding the meaning of the rest of the bytes is left as an exercise
for the reader, but I draw your attention to the 2nd byte, which is 1F.
This byte indicates which RF bands are physically supported by the MS
(mobile station) hardware, and 1F means quad-band, i.e., all 4 of 850,
900, 1800 and 1900 MHz.  Thus even though my trusty GTA02 is only
900/1800/1900 MHz tri-band in reality, it believes itself to be
quad-band!

Digging some more, one finds that the 16 bytes quoted above appear in
the moko10 and moko11 fw images (convert them from *.m0 to plain binary
with the mokosrec2bin.c utility I wrote almost a year ago, then do the
binary grep with the memmem() C library function), and further
analysis reveals that these standard firmwares unconditionally
overwrite the /gsm/com/rfcap file in FFS with the hard-coded string
of bytes on every boot.  To convince yourself of the latter fact, take
a GTA02 modem with moko11 in it, change the rfcap file in FFS to
something else, reboot the modem normally, and observe that the rfcap
file will be reverted back to the 16 bytes shown above, claiming to be
a quad-band GSM device.

My leo2moko firmware does not contain this rfcap-resetting feature:
it does not automatically overwrite the rfcap file with anything, and
uses whatever settings happen to be written in the FFS (the modem's
flash file system).  At the present, there is no practical difference:
if your modem ever ran moko10 or moko11 prior to being flashed with
leo2moko, the content of the /gsm/com/rfcap file in FFS will be what
moko10/11 wrote into it the last time it booted, which is the hard-
coded I am quad-band value.

But I wonder - and this is really the main reason for this lengthy
post of mine - is it really a good idea for a tri-band GSM modem to
believe itself to be quad-band?  There are two potential problems I
can think of:

1. 

Re: /gsm/com/rfcap: tri-band GSM modem believes itself to be quad-band

2014-01-15 Thread joerg Reisenweber
On Thu 16 January 2014 02:58:29 Michael Spacefalcon wrote:
 Hello Om community,
 
 In the course of hacking TI GSM firmwares, I have come across something
 that some of you may find interesting, or might even have some insight
 into.
 
 We all know that our good familiar Neo Freerunner (GTA02) was made in
 two versions: one with 900/1800/1900 MHz bands, the other with
 850/1800/1900 MHz instead.  The hardware difference is one RF SAW
 filter part populated differently on the same PCB; see this picture:
 
 http://wiki.openmoko.org/wiki/File:Gta02a6_comms_chips_under_shield.JPG
 
 The SAW filters for the GSM downlink Rx path are the 3 little buggers
 near the upper right corner, immediately adjacent to the shiny metallic
 component which is the antenna switch.  There is one Rx SAW filter for
 each of the 3 supported bands: one for 1800 MHz (both GTA02 versions),
 one for 1900 MHz (ditto), and one populated for either 850 or 900 MHz.
 (*I think* the topmost one out of the 3 in that picture is the one
 responsible for the 850 vs. 900 MHz difference, but please double-check
 that before attempting any surgery on your Neo!)
 
 Well, here is the part which will surely surprise at least some of you:
 the standard firmware for the GTA01/02 modem (which is the same for
 all versions, both GTA01 and GTA02) does not actually know which 3
 frequency bands are supported by the device it runs on!  And no, it
 does not auto-detect either: there is no way (short of ESP) for any
 firmware running on the Calypso/Iota/Rita chipset to divine what kind
 of SAW filter sits between that chipset and the antenna.
 
 Instead, as strange as it may sound, the modem (at least when running
 the standard mokoN firmware, see below) believes itself to be quad-
 band!
 
 In TI's universe, the standard way to teach a GSM device (phone or
 modem) which GSM frequency bands it supports is *not* to hard-code
 that knowledge in the firmware at compile time; instead this property
 is stored in a configuration file named /gsm/com/rfcap in the GSM
 device file system.  Yes, TI-based GSM devices all use a flash file
 system with a very UNIX-like look and feel, including UNIX-style
 pathnames; see my write-up:
 
 https://bitbucket.org/falconian/freecalypso-sw/src/1852900ce9ea4ac52d4648f7
 d9ca46897eb3640b/doc/TIFFS?at=default
 
 Like most files in TI's GSM FFS, /gsm/com/rfcap is a binary file, not
 ASCII.  It is a file of exactly 16 bytes, and although I haven't found
 a formal document describing its format in plain English, we can study
 the code that reads this file and acts upon its content:
 
 http://scottn.us/downloads/peek/TCS3.2_N5.24_M18_V1.11_M23BTH_PSL1_src/g23m
 -gsm/rr/rr_csf.c
 
 The 16-byte file is being read into a variable of type EF_RFCAP, which
 is defined here:
 
 http://scottn.us/downloads/peek/TCS3.2_N5.24_M18_V1.11_M23BTH_PSL1_src/g23m
 /condat/com/include/pcm.h
 
 Lines 442 through 460 (inclusive) give the structure definition, which
 is followed by the definitions for the bit fields in each byte.
 
 And here is what this /gsm/com/rfcap file contains on a standard GTA02
 modem, as revealed by a TIFFS parsing tool such as the mpffs-tools-r1
 package I released last summer:
 
 00 1F 41 14 00 00 00 00  50 00 00 A5 05 00 C0 00
 
 Decoding the meaning of the rest of the bytes is left as an exercise
 for the reader, but I draw your attention to the 2nd byte, which is 1F.
 This byte indicates which RF bands are physically supported by the MS
 (mobile station) hardware, and 1F means quad-band, i.e., all 4 of 850,
 900, 1800 and 1900 MHz.  Thus even though my trusty GTA02 is only
 900/1800/1900 MHz tri-band in reality, it believes itself to be
 quad-band!
 
 Digging some more, one finds that the 16 bytes quoted above appear in
 the moko10 and moko11 fw images (convert them from *.m0 to plain binary
 with the mokosrec2bin.c utility I wrote almost a year ago, then do the
 binary grep with the memmem() C library function), and further
 analysis reveals that these standard firmwares unconditionally
 overwrite the /gsm/com/rfcap file in FFS with the hard-coded string
 of bytes on every boot.  To convince yourself of the latter fact, take
 a GTA02 modem with moko11 in it, change the rfcap file in FFS to
 something else, reboot the modem normally, and observe that the rfcap
 file will be reverted back to the 16 bytes shown above, claiming to be
 a quad-band GSM device.
 
 My leo2moko firmware does not contain this rfcap-resetting feature:
 it does not automatically overwrite the rfcap file with anything, and
 uses whatever settings happen to be written in the FFS (the modem's
 flash file system).  At the present, there is no practical difference:
 if your modem ever ran moko10 or moko11 prior to being flashed with
 leo2moko, the content of the /gsm/com/rfcap file in FFS will be what
 moko10/11 wrote into it the last time it booted, which is the hard-
 coded I am quad-band value.
 
 But I wonder - and this is really the main reason for this 

Re: /gsm/com/rfcap: tri-band GSM modem believes itself to be quad-band

2014-01-15 Thread Michael Spacefalcon
joerg Reisenweber jo...@openmoko.org wrote:

 Bottom line: even 850MHz FR can do 900MHz but the reception is pretty poor=
 though just sufficient in usual urban environment.

Hmm, interesting!  Of course given the geographical theater I operate
in, my interest would be in going the other way around.  The funny/sad
thing is, I do not currently have *any* GSM (or GSM+newstuff) device
which has proper support for the 850 MHz band (the secondary GSM
band in the lands I operate in) and which I can hack in any way at all:
sure, there are 3G+ smartphones all around me, and they all claim to be
quad-band when they go into GSM fallback mode, but of course they are
the typical Android etc crap, and I don't know how to hack them (if
that can be done at all), so I don't even know whether that SGS2 (as
an example) is connected to an 850 MHz network or a 1900 MHz one!

As to the hackable Calypso devices I have (one FR and a few Pirelli
DP-L10s), they are all 900/1800/1900 MHz, so to this day I don't even
know what GSM services might exist in the 850 MHz band around me...
I do use one of these devices as my everyday phone though (and the
non-Calypso Mot V66 I used previously was/is 900/1800/1900 MHz as
well), so basically all my cellphone-using life I've been using what
is effectively a 1900 MHz only device.  I roam over a pretty wide area
and have yet to encounter a spot without good coverage - and it is
kinda neat when I realize that I get all this great coverage despite
my phone supporting just GSM1900 and absolutely nothing else!
(Meaning nothing else relevant to my geographical area.)

But given what you just said, I am now tempted to try running cell_log
(from OsmocomBB) set to the [128,251] ARFCN range (corresponding to
the 850 MHz band), and see if it can pick anything up.  I'll try it
when I get some time to play.

 I don't think network/BTS can direct the mobile to another channel, it can
 only adveritse other channels but it's up to mobile to determine if that=20
 channel actually works in the particular situation

Indeed I think that the scenario I hypothesized in my original post is
very unlikely in reality, but as far as the network directing a mobile
to a channel, I was thinking along the lines of what happens in call
establishment, channel hopping etc - granted, I still have a lot of GSM
learning ahead of me as I set out to reintegrated that code from
http://scottn.us/downloads/peek/ in the place of the binary libs in
leo2moko, but I was under the impression that whenever the BTS basically
tells a mobile let's continue this conversion on channel so-and-so,
that direction contains not only the new timeslot number, but also the
new ARFCN, and at least in theory the network is free to pick any ARFCN
which it serves and believes the mobile to support as well.

But of course I could be entirely off-base here, as I haven't really
learned that part of GSM yet.

 there may be other=20
 problems to receive 900MHz band than just a SAW filter, like local interfer=
 ence=20
 or whatever.

I see what you are saying: the scenario of a tri-band mobile device
advertising and believing itself to be quad-band (what Om modems do)
is indistinguishable from that of a genuinely quad-band device having
the 4th band rendered unusable by some unpredictable external factors
like local interference.  A valid point indeed.

The fact that, as you say, an FR made for 850 MHz could receive 900 MHz
and work so-so (limp along) does make for an argument in favor of
keeping the I am QB setting in that /gsm/com/rfcap file - if that
file were changed to reflect just the bands officially supported by
a given FR unit, it would not even attempt tuning its radio to the 4th
band.

Has anyone else in the community ever used an FR successfully in the
wrong band?

 When the mobile can't use a certain frequency, the BTS will not
 force the mobile to that frequency.

OK, I need to learn some more about what happens in channel hopping
scenarios, and how (and by whom) the frequency channel is selected in
those cases.

Oh, and while we are on the subject of GSM frequency bands, I guess I
need to give everyone an update on my efforts to build my own quad-band
Calypso phone.  Right now that project is bogged down in mechanical
design issues which have nothing to do with the number of GSM bands or
even with the Calypso chipset.  What I really want is something as
nearly-identical as possible to the Pirelli DP-L10, and I won't feel
satisfied with anything less than that.

Back in 2013-11 I was in communication with some seller (German
apparently, but advertised on a Chinese site) who claimed to have
around 700 of those Pirellis for sale.  I wanted 100, and the price
would have been some 3-4 months worth of my spare budget.  But just as
I raised the money for the first partial order, the guy told me that
someone else bought all of those phones, they had no more left, and
whoever bought all 700 or however many they had, was not willing to
resell any of them