Hi, Aaron,
Always good comments. Here is my response.
1. Code churn: it's always what you get what you ask for. We have experienced this situation before. I know that for LOC, "**" and "**/*.java" means the same thing, since LOCC only sends java sizes. People tend to use "**", and take it for granted that it's for java code. There is really nothing I can do here.
2. More detailed information about telemetry and chart parameters.
I concur that this is a very important usability issue.
With client side java script, it's possible to provide the same level of information as other analyses. We visited this problem long ago, and the decision was we want to support those guys who run browsers with javascript turned off.
In fact, I am quite good at writing client side java script. If everyone thinks now it's the time, I have no problem adding this functionality.
The good news is that all current public telemetry reports require no parameter.
3. Standard deviation line: (a) So far we don't understand how statistical procss control can be applied to telemetry. I think this is one of the future researches of telemetry, perhaps another Ph.D. thesis.
(b) It's bad to hard code any arbitrary information in our analyses, including standard deviations. In fact, Christoph talked to me months ago that he wanted more operators. As a generic approach, we can allow operators to be plugged in, including standard deviation operator.
current implementation of operator is:
newStream = stream1 + stream2 The + operator can also be written (not in current implementation)
newStream = ADD(stream1, stream2) If we have a StdDev operator
newStdDevUpStream = StdDev(stream1, +1)
newStdDevDownStream = StdDev(stream1, -1) Then you can have a chart containing
(newStdDevUpStream, stream1, newStdDevDownStream)Now see, you can not only plug in reduction functions, but also operators. Cool idea, eh?
The bad news is that I am not convinced it will add much value to my thesis, so it will be low priority even if I decide to do it.
Cheers,
Cedric
Aaron Kagawa wrote:
Hey Cedric,
I just ran the Churn-Member-Relative-Chart and Churn-Member-Relative-Chart for hacky2004-all from Jan 2, 2005 to the current week (see the bottom of this email). And there are some really strange results. I have some outrageous Code Churn around 400,000 to 500,000 lines of code compared to the rest of the hacky2004-all developers, who seem to average in the tens of thousands. Is that a bug? Or am I really that good? :)
Haha.. Ok, I just spent a little more time figuring why the heck I have so much Churn. It turns out that I commit HTML documents for documentation that lives in hackyPRI. These HTML doc files are rather large. It turns out that I have to change the Parameters values to "**/**.java, true", to get the churn on java code. After fixing that, Christoph (with ~61K Absolute Churn and ~21K Relative Churn) seems to be doubling almost everyone else's churn numbers. Anyway, remember to use "**/**.java" on the telemetry wall for Churn Telemetry Streams.
A couple more issues: - in the analysis Telemetry Chart Analysis, is the Parameter Values selector going to allow a default parameters by leaving this text field blank? Currently, I have to type in something. Often, it is a little difficult to guess what Parameters are needed. Especially for parameters like "mode". The error messages help. But, it usually takes me a couple of guesses to correctly enter in complex parameters (mode, filePattern, isCummulative). Furthermore, I'm not sure that a non-csdl member knows what they have to type in for a filePattern or mode parameter. For example, I tend to enter in "**.java", which is wrong, the correct string is "**/**.java". It would be interesting to see how many times Hackystat users executed the Telemetry analyses.
- it is a little difficult to track down the descriptions of these Reports, Charts, Streams, and Reducers. If I'm correct the only Telemetry "thing" that has a description is the Reducer. So, what I had to do to figure out what a Report was made of (from a top down approach) was go to the Telemetry Management page for the Reports, Charts, Streams, and Reducers reading the Names and Definitions. Not to mention, that a Report can consist of one-to -many Charts, a Chart can consist of one-to-many Streams, etc.. Even for me (a person that knows about Telemetry Reducers, Streams, Charts, Reports), the way the information is presented made it hard for me to understand what I'm seeing and how it was calculated.
Management of the Telemetry (primarily for the administrator) is good in a bottom-up fashion, like what you have in the Telemetry Management preferences. But, for a user to understand the results its probably better presented in a top-down fashion. Another command that shows the underlying telemetry pieces for a given Report, Chart, and Stream would be a useful addition. For example, the use could select "DevelopmentProgressReport " then get a listing of the Charts, Streams, and Reducers (including descriptions) that were used to build the Report.
Furthermore, understanding telemetry has the disadvantage of being dynamic, because other traditional commands can provide the "more" documentation string to explain the analysis.
- what ever happened to the Standard Deviation lines?
thanks, aaron
<Series name="Churn-Member-Relative<**, true>:[EMAIL PROTECTED]"> <Category name="02-Jan-2005" value="0" /> <Category name="09-Jan-2005" value="0" /> <Category name="16-Jan-2005" value="0" /> <Category name="23-Jan-2005" value="2368" /> <Category name="30-Jan-2005" value="61026" /> <Category name="06-Feb-2005" value="61586" /> <Category name="13-Feb-2005" value="62306" /> <Category name="20-Feb-2005" value="104487" /> <Category name="27-Feb-2005" value="105089" /> <Category name="06-Mar-2005" value="107093" /> <Category name="13-Mar-2005" value="123551" /> <Category name="20-Mar-2005" value="264958" /> <Category name="27-Mar-2005" value="265278" /> <Category name="03-Apr-2005" value="336756" /> <Category name="10-Apr-2005" value="418430" /> <Category name="17-Apr-2005" value="418430" /> </Series>
<Series name="Churn-Member-Absolute<**, true>:[EMAIL PROTECTED]"> <Category name="02-Jan-2005" value="0" /> <Category name="09-Jan-2005" value="0" /> <Category name="16-Jan-2005" value="0" /> <Category name="23-Jan-2005" value="2934" /> <Category name="30-Jan-2005" value="62196" /> <Category name="06-Feb-2005" value="64046" /> <Category name="13-Feb-2005" value="65712" /> <Category name="20-Feb-2005" value="108587" /> <Category name="27-Feb-2005" value="109533" /> <Category name="06-Mar-2005" value="117757" /> <Category name="13-Mar-2005" value="136135" /> <Category name="20-Mar-2005" value="278504" /> <Category name="27-Mar-2005" value="278900" /> <Category name="03-Apr-2005" value="403368" /> <Category name="10-Apr-2005" value="487308" /> <Category name="17-Apr-2005" value="487308" /> </Series>
