On Tue, Aug 11, 2009 at 8:34 PM, Razvan Musaloiu-E.<[email protected]> wrote: > Hi! > > On Tue, 11 Aug 2009, Philip Levis wrote: > >> >> On Aug 7, 2009, at 1:27 PM, Razvan Musaloiu-E. wrote: >> >>> Hi! >>> >>> About 2 months ago, a commit [1] to CpmModelC.nc that eliminates the >>> problem of receiving packets (non-zero PRR) for negative SNR also made a >>> significant change to the hardcoded PRR/SNR curve. This was not adequately >>> documented in the commit message and many people might not be aware of >>> this. Here is a plot that shows the differences: >>> http://bit.ly/znrwQ > > I made a mistake in the above graph. In Yin's data the SNR is actually > SINR. After accounting for this the new graph looks like this: > http://farm3.static.flickr.com/2456/3812438521_c08b3ec98e_o.png > > So the new curve is actually in agreement with Yin's data. :-) > >>> The points represent the data collected by Yin Chen, a member of our lab. >>> In his tests the noise level was controlled by using a noise generator so >>> it covers a wide range. Similar data was also reported by the following >>> paper: >>> http://portal.acm.org/citation.cfm?id=1460427 >>> > > The conversion of SINR to SNR should also make this data in agreement with > the new curve. > >>> So there are reasons to believe that even the current curve we have in the >>> tree is not the most accurate one. Caution is advised! :-) > > Considering that multiple datasets collected in very different ways are in > agreement now I think there are good reasons to believe the new curve is a > good one. > >>> [1] >>> http://hinrg.cs.jhu.edu/git/?p=tinyos-2.x.git;a=commit;h=afe96d7e2a3747bba450e3db88a89569a0ba53b6 >> >> The methodology Yin used to collect the data set in the above figure is not >> appropriate for calculating SNR/PRR curves. > > I think it is. Here is a visual representation of the signal/noise space > explored by Yin's data: > http://farm4.static.flickr.com/3548/3812438643_bf272d9b56_o.png > > Only 12 receiving motes were used but multiple TX levels were used. > Here are two other views: > > SNR vs Signal > http://farm3.static.flickr.com/2448/3813253680_f72943d1a3_o.png > > SNR vs Noise > http://farm3.static.flickr.com/2623/3813253658_12aa32c68a_o.png > >> There are three separate issues, all of which have a common concern: >> averaging. While the data sets in the above paper are suitable for its >> conclusions, they have problems when it comes to simulation. >> >> 1) It combines many node pairs: each pair can have a different SNR/PRR curve. >> This is why an SNR of 3dB can have a PRR of 95% or 5%. > > Here is the SNR vs PRR for each node: > http://farm3.static.flickr.com/2607/3812438555_0199f0591d_o.png > > It seems that the overall shape of the curve stays the same and is only > shifted to the left or right. We should be able to simulate this using an > appropriate random variable. More experiments are needed to clarify > this. > >> 2) Signal is a not a controlled variable, and it is averaged. A lack of >> control means that it will vary due to environmental conditions, and the fact >> that it is averaged over received packets leads to sampling bias. It is not >> clear in the paper if the plot averages the RSSI of packets along a link or >> averages the per-packet SNR. Since N is considered to be constant (more on >> that below), the two are equivalent. But if the signal strength has >> variation, and you observe the SNR only of received packets, you can observe >> an SNR higher than the PRR would suggest. >> >> 3) Noise/interference are not controlled. Taking the average of N assumes >> that it is a gaussian variable; while N is, external interference I can >> disrupt this measurement. This might explain why the SNR/PRR curve is lower. >> E.g., if 10% of my packets have a high interference, my averaged N will go up >> significantly in a way that will lead the SNR curve to lead to incorrect >> conclusions. > > Yin has some plots that shows the variations of the signal and noise in > his data. I agree that both are legitimate concerns.
Just to clarify a bit, the experiment used channel 26, and was conducted in a quiet office. So basically both the signal and noise RSSI were quite stable. Each data point in the SNR vs PRR graph is averaged over 250 packets. And for 90% of the data points, the standard deviations of the RSSI across the 250 packets is smaller than 0.8 dB, and almost all data points have std below 1 dB. - Yin > >> There are ways to give a sense of the degree of these methodological >> limitations. The spread of discrete points in the Figure is a good approach >> for 1). A plot of the S distribution could help with 2). A plot of the N >> distribution could help with 3). >> >> The current curve in TOSSIM was generated from a data set that Kannan >> collected using a variable attenuator, shielded cabling, and 2 micaZ motes. >> I've attached a plot of the associated data. The error bar along the X-axis >> is the standard deviation of signal strength. Using a variable attenuator and >> shielded cables leads these values to be very stable. The red bar shows the >> noise floor, which he calculated as the mode of RSSI readings taken when >> there were no transmissions. Since these were measured in a closed system, >> they are not affected by external interference. > > I did not know how was the data collected by Kannan. From your description > I understand that the noise was always constant (-96 dBm). Is this > correct? It also seems that in the experiment tested only a very small > number of distinct PRR values (I can only see only 4-5). It could be > possible to test a few more? More points in the -90 and -93 dBm would > paint a much clearer picture. > >> The prior curve was generated with an inferior data set, taken from an >> uncontrolled environment (such as Yin's). > > I think the experiments did by Yin are complementary to the one did by > Kannan because they explore the SNR when the noise is different from the > noise floor. I think this is very important because it's not so obvious > that the SNR vs PRR curve is the same for various noise levels (which is > exactly what CPM is relying on :-)). > > To reiterate what I said somewhere in the middle of this reply: the (at > least visual) agreement between three very different experiments (Kannan, > the Sensys 2008 paper and Yin) is an excellent indication that the new > curve is a good one. :D > > All the best! > Razvan ME > _______________________________________________ > Tinyos-help mailing list > [email protected] > https://www.millennium.berkeley.edu/cgi-bin/mailman/listinfo/tinyos-help > _______________________________________________ Tinyos-help mailing list [email protected] https://www.millennium.berkeley.edu/cgi-bin/mailman/listinfo/tinyos-help
