Re: Three different databases for gsm celltower locations
Joerg Reisenweber wrote: Am Sa 4. April 2009 schrieb Stefan Schmidt: OpenBmap likes to have more fields for quality informations about GPS and GSM. I strongly suggest to include fields for TimeAdvance values for each BTS. TA is a much better and more reliable value for distance to Base Transmitter Sation than dB signal strength. See: http://lists.openmoko.org/pipermail/openmoko-kernel/2008-April/002434.html and http://lists.openmoko.org/pipermail/openmoko-kernel/2008-June/002987.html I see no problem with storing the TA, to allow testing of algorithm using this (and thanks for bringing that to my knowledge). Some precisions about openBmap approach. We do not try to locate or use location of the BTS by itself. We try to build a coverage map of every cell. What is the area where we have been connected to the cell. Then we try to find a position for this area which makes sense (e.g. barycentre). We do not so far use the signal strength for this. We thought about using it nevertheless for positioning afterwards: get position of the serving cell, plus neighbours which have signal strength strong enough to make sense, and try to correlate all of this to refine the final position. Onen ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Please excuse the late reply... On Mon, 2009-04-06 at 21:53 +0200, Onen wrote: Hallo Jan, I have seen the last commits from you about adding GSM based localisation. First this is great, that it reaches the framework! This brings me some questions: 1. This has been especially a surprise to us, as we at openBmap have currently been thinking how to bring this to the users. We already did some work on exporting the database under SQLite files for every country. And on top of this, a small DBus service to bring the localisation to everyone... I hacked this together in some spare hours while i was in Bern for OpenExpo, so this is in no way the final code or interface. I just wanted to try out the data i've been collecting with Cellhunter (BS conenction). The cell db is currently exposed via ogsmd. The location interface should be distinct from that. I've been talking with Daniel about this for some time: * each client should request an accuacy and an update frequency, the location daemon will then choose the an approach to find the location (cells/wifi, GPS) * maybe also some notifications for locations I am no hacker, but still would have hoped to bring some help on this... If you are interested, how would you like to proceed? 2. I had a look at the code, and suppose you are building your database out of the data from CellHunter. Am I correct? Yes. The algorithm i use is extremly simple: I just calculate the boundingbox of all measurements for a given cell and then store the center and radius in a binary file. When calculating the position, i just average all found positons, or if none a found, take the center of the LA's bounding box. I think CellHunter is a great project but we still think that we should log HPV-Dops, speed, in order to be able to filter low quality measures. This is currently discussed on another thread on this list [1]. If you combine a high speed with low Dops, you end up with very unprecise positions for building the cells coverage, don't you think? For now even the signal strength is not used in my code. Still, it gives an accuracy that should be useful for AGPS. I would like to know what is your opinion about all of this. Are you interested in our approach, and are you interested in taking in consideration in your code the extra fields our data contains (Dops, speed, software id, software version, etc.) to possibly filter measures to keep only higher quality? I would propose to include even more data: * some GPS receivers can give an inaccuracy in cm in addition to DOP, this even includes position errors from reflections, fading and not just the satellite position geometry * speed and *direction*, the handover algorithm used by operators take these into account, so we should collect them * if we are in a call or have a GPRS * if available, log the timing advance * don't log each neighbour seprately, but upload a measurement report with time, position and *all* current cells (serving + neighbours). also log what the max amount of tracked neighbours is. then we can deduce which cells are *not* visisible at that position. this should give us better accuracy by using signal shadowing. * also allow logging of different radio technologies (GSM, UMTS, Bluetooth, Nearfield/RFID, WiFi b/g, ...) with the same measurement report. Also please take a look at: http://lwn.net/Articles/304218/ and http://lwn.net/Articles/325016/ -- Jan Lübbe jlue...@lasnet.dehttp://sicherheitsschwankung.de gpg-key 1024D/D8480F2E 2002-03-20 fingerprint 1B25 F91F 9E7B 5D4F 1282 02D6 8A83 8BE4 D848 0F2E ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
On Tue, 2009-04-07 at 23:49 +0100, Joerg Reisenweber wrote: Am Sa 4. April 2009 schrieb Stefan Schmidt: OpenBmap likes to have more fields for quality informations about GPS and GSM. I strongly suggest to include fields for TimeAdvance values for each BTS. TA is a much better and more reliable value for distance to Base Transmitter Sation than dB signal strength. See: http://lists.openmoko.org/pipermail/openmoko-kernel/2008-April/002434.html and http://lists.openmoko.org/pipermail/openmoko-kernel/2008-June/002987.html As Jörg said, the timing advance is only valid while a channel to the BTS is open. An active GPRS connection is enough, but the TA for that is sent in the EM command for GPRS. When it is invalid 255 is sent, otherwise the actual value. You may need to create traffic to see a vaild TA here. The TA field for voice calls uses '0' for invaild, so it's not as nice for getting proper readings. -- Jan Lübbe jlue...@lasnet.dehttp://sicherheitsschwankung.de gpg-key 1024D/D8480F2E 2002-03-20 fingerprint 1B25 F91F 9E7B 5D4F 1282 02D6 8A83 8BE4 D848 0F2E ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hello. On Sat, 2009-04-04 at 02:42, Thomas Landspurg wrote: The good news is that we are communicating and discussing while progressing slowly. With Sebastian of CellHunter, we basically agree that Cell Hunter would upload is cells to OpenCellID while maintaining their own database for the purpose of the game. OK With Onen one of the idea (in my view, but Onen to confirm ) is that to enable a the GPS Logger to be connected to OpenCellID to upload cell informations directly. Now, the main part remains the positionning of OpenBMap. The main difference (in my view) between OpenBMap and OpenCellID is that OpenBMap is more focused on getting accuracy for cells , while we (OpenCellID) are more concerned about having the biggest and complete coverage. Having a good coverage is as important as having quality measurements. What is about these fields in the db to store informations about the quality. Are you willing to add them? What about wifi accesspoints? Are you willing to add such things, too? regards Stefan Schmidt ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hello. On Sat, 2009-04-04 at 12:02, Sebastian Hammerl wrote: like Thomas said I will submit the data of cellhunter to opencellid while keeping my own db for more game data. So opencellid will be the main source for cell information and I think this is the best solution because it is just the largest db. Congrats to thomas for the huge growths in the last weeks. Wrote it before. Good coverage is great, but not if we buy this by letting some quality data and possibilities like wifi APs aside. If this could be added to openceelid I aggree that openbmpa should merge their db into it and use that one as basis for the webservices they use. At this time I do note have the time to submit the data because I have to finish my master thesis until mid of may. But after that I promise to upload new cellhunter cells periodically. Please also integrate Jan's patches for newer FSO releases when you reach the free-time slot. A FSO approach for getting cell information should query opencellid. Perhaps Thomas should think about to transfer the db to a faster server I have some troubles sometimes to reach the site. Perhaps we should set up a mirror at the openmoko servers or something else. But these should be problems for the future. I think we should prefer a way to have the db on the device itself. Onen gave some hints why. That of course includes a way to sync new data to the device. regards Stefan Schmidt ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hello. On Sat, 2009-04-04 at 15:24, Onen wrote: Sebastian Hammerl wrote: I will talk from my part the client code. I know you have little time right now for a collaborating project. But I think we have much better to do than duplicating this aspect. If you would agree on coding on the same soft, we would save a lot of time. I am thinking about internationalisation of the code, etc... While talking about the client code. Sebastian, what do you think about the idea to adding wifi AP's to your game concept? Same db, perhaps some a bit different schema for the game-points. regards Stefan Schmidt ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hello. On Sat, 2009-04-04 at 15:03, Onen wrote: Another point is the other signals than GSM. Our initial vision was to build a database of communicating objects, WiFi, Bluetooth, etc... Can anybody give me some ideas how BT should be useful here? From what I know there are very little fixed BT accesspoints around. Likely to be killed by wifi anyway. I associate with bluetooth something that is mobile itself therefor not useful to get an idea rom your current location from. May be good to add Jan to the thread, as he has started work on the databases? Jan should be on the ml's already. regards Stefan Schmidt ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hi, Stefan Schmidt wrote: On Sat, 2009-04-04 at 15:03, Onen wrote: Another point is the other signals than GSM. Our initial vision was to build a database of communicating objects, WiFi, Bluetooth, etc... Can anybody give me some ideas how BT should be useful here? From what I know there are very little fixed BT accesspoints around. Likely to be killed by wifi anyway. I associate with bluetooth something that is mobile itself therefor not useful to get an idea rom your current location from. Well maybe Bluetooth was not the best example. I wanted to put the emphasis that the original idea of openBmap was to get a map of all communicating objects (RFID, future things, ...). Nevertheless here are some examples of static BT points we may encounter pretty soon: * some experiments have been carried on in Paris subway about BT beacons which help blind people find their ways in the labyrinth. They have a special device at the ear, which says where they are, and if they go in the wrong direction, etc... * ad panels in the subway, or on the street, may have bluetooth to distribute details to phones * in Paris, there are BT stations on the street, you can get connected to with your phone, and download maps and or details of the area you currently are. A reason why BT may still last over WiFi may be the power needed to have BT on in comparison of WiFi, don't you think? If anybody sees more examples... Onen ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Onen onen...@free.fr writes: A reason why BT may still last over WiFi may be the power needed to have BT on in comparison of WiFi, don't you think? What about digital radio broadcasting stations or RDS information from regular radios? -Timo ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Timo Juhani Lindfors wrote: Onen onen...@free.fr writes: A reason why BT may still last over WiFi may be the power needed to have BT on in comparison of WiFi, don't you think? What about digital radio broadcasting stations or RDS information from regular radios? These are very interesting ideas! I have no idea how this all works, what is the coverage of a broadcasting antenna, and if this is possible to differentiate them through the received signal. Plus we would need a working USB radio receiver. Onen ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hi, Thomas Landspurg wrote: The good news is that we are communicating and discussing while progressing slowly. With Sebastian of CellHunter, we basically agree that Cell Hunter would upload is cells to OpenCellID while maintaining their own database for the purpose of the game. With Onen one of the idea (in my view, but Onen to confirm ) is that to enable a the GPS Logger to be connected to OpenCellID to upload cell informations directly. For sure this is a possibility, either direct upload to OpenCellID, or as CellHunter plans to do, regular upload of new data (but for the moment I may like less the second, because it would mean we would still have many databases ;-), I still hope to see some merging). Now, the main part remains the positionning of OpenBMap. The main difference (in my view) between OpenBMap and OpenCellID is that OpenBMap is more focused on getting accuracy for cells , while we (OpenCellID) are more concerned about having the biggest and complete coverage. This is absolutely correct. I have the exact same opinion. That is the reason behind keeping more details about measures. This allows to gather a lot of data, but with time, we can trash the low quality ones, because we have got high quality ones since. This brings three questions: 1. if you have big HPV-Dops, your position is not very precise. If you add to this that you have a high speed, then when you take your measure, the GPS position is very inaccurate. And the time you get notified that the GSM connection has changed, this adds to inaccuracy. My question is: do people think this argument makes sense? 2. Do OpenCellID or CellHunter think this could be possible to add these extra fields to their database, and measures? This would allow to use inaccurate data, until when we have better ones. Then we could filter the low quality measures. I think we are still all learning a lot, and this would imply that extra fields could be added in the future. So this is probably not only a one shot change. 3. The database should also keep track of the software (id and version) which has logged the data: this allows to ignore/remove data which has been submitted by a buggy software, even if the bug is discovered much later. That is also the idea behind keeping the GSM chip model + Firmware version + GPS chip, etc... It there was a way to export cells informations, I would be happy to import them to the main OpenCellID database, I can suggest to OpenBMap for instance to use OpenCellID interface for non existing cells in their database? The idea behind the license of the data is exactly to allow this. I think that once we have clarified the situation, we can automate this. Lastly some update of OpenCellID: we are now more than 210 000 cells ( http://www.opencellid.org/cell/stats ) and 19 000 000 of mesures which is already great, so having support of the FSO would definitively gives the final boost to the project. So thanks for the support for evrybody. Yes we have followed the growth of your database. That's very good! Do not forget CellHunter which has grew over 2 000 000 measures! Congratulations to both of you! Another point is the other signals than GSM. Our initial vision was to build a database of communicating objects, WiFi, Bluetooth, etc... Thomas do you intend to keep strictly GSM data in OpenCellID? Sebastian same question with CellHunter? May be good to add Jan to the thread, as he has started work on the databases? Onen ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hi, Sebastian Hammerl wrote: At this time I do note have the time to submit the data because I have to finish my master thesis until mid of may. But after that I promise to upload new cellhunter cells periodically. At this momemt there are about 4 cells with over 200 measures. At least germany is covered very well. Congratulations, that's very good! We are over 1 cells, without logging neighbours, so direct comparison is biased ;-) There is much work which is duplicated between OCI, CH and OBM: * KML, etc... map on the website * the website by itself * the promotion we do to get new users * the server code * the client code I will talk from my part the client code. I know you have little time right now for a collaborating project. But I think we have much better to do than duplicating this aspect. If you would agree on coding on the same soft, we would save a lot of time. I am thinking about internationalisation of the code, etc... From my side, I think the only thing I need to be sure, is would you accept code which reduce the amount of data, in preference of quality (no log in a call, no log when too high speed, etc...). The time saved could allow to start other platforms support (Thomas there is no Android client, right?). My motivation here is only the will to prevent duplicating effort. Let me know what's your opinion. A FSO approach for getting cell information should query opencellid. Perhaps Thomas should think about to transfer the db to a faster server I have some troubles sometimes to reach the site. Perhaps we should set up a mirror at the openmoko servers or something else. But these should be problems for the future. I disagree here. The database should definitely be located on the phone itself. Which does not diminish the interest of a Web API (for mashups of all sorts). Advantages: * privacy: you get your location without involving third party * efficient: no problem of server scalability, lag, etc... * no need of a data connection * update can be done from time to time while connected to WiFi/USB etc... Onen ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hi, Stefan Schmidt wrote: I can see some technical problems you may face here. Cellhunter needs some db fields for the group management and ranking. OpenBmap likes to have more fields for quality informations about GPS and GSM. Different layouts of the databases, etc. What should work easily is that the fields of the cellhunter db that are not relevant to the group management get synced to opencellid and openbmap. Then these both have to figure out a way how to sync with the different ideas of stored data for position quality. That one is of course harder. Best would still be to join forces and go for one central collection point. I don't see this happen tho. So please, step back and think about how you are able to sync for efforts at least. At first glance I see: * OCI supports the extra fields of OBM, we can store our data there. (I see a difficulty here, is that we may discover with time additional interesting information to collect, and maybe Thomas is not very keen on the idea of modifying too often his database...?) * CH and OBM merge. CH collects already extra fields for the game. So we would add the extra fields for OBM. * another approach I still have to find out ;-) I need some thinking about it to be innovative... Freesmartphone, which will hopefully a big consumer of this in the future, also would like to see a effort to collect the position of wifis and use them in combination with GSM for better accuracy outdoor and also in buildings where no GPS is possible at all. We could also imagine Bluetooth. This is the initial vision of openBmap (all the communicating objects). I'm working on the networking part of FSO to give the logging clients an easy interface for the wifis around you. That's cool :-) And please, before answering here. Step back and think about it. I don't want flames but constructive work here. :) As said before, I will keep trying to find other ways which would bring everybody to an agreement... Onen ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
Re: Three different databases for gsm celltower locations
Hi, On Sat, 4 Apr 2009 00:05:43 +0200 Stefan Schmidt ste...@datenfreihafen.org wrote: I'm wondering what happended to the discussion about three different databases available. Back at the last time this topic came up I felt that there will be ongoing work to avoid this concurency and try to get the databases merged. thanks for bringing that up, it's quite sad, actually. Best would still be to join forces and go for one central collection point. I don't see this happen tho. So please, step back and think about how you are able to sync for efforts at least. One other possibility if this isn't possible for one reason or the other is to aggregate the individual databases into one big database. The licenses would allow for it and opencellid as well as cellhunter export the raw data (Openbmap's Offline API will be available during first semester 2009 according to their website). I would think of this as purely a generated database without any upload functionality to avoid yet another cell location database project pulling into a fourth direction. Freesmartphone, which will hopefully a big consumer of this in the future, also would like to see a effort to collect the position of wifis and use them in combination with GSM for better accuracy outdoor and also in buildings where no GPS is possible at all. Right, during the OpenExpo in Bern Jan hacked something together to see how well GSM locating would perform using the cellhunter db and accuracy was quite okay (position was about 200-500m off). Regards, Daniel Willmann signature.asc Description: PGP signature ___ smartphones-standards mailing list smartphones-standards@linuxtogo.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/smartphones-standards
