Aaron brings up so much stuff that it's going to take multiple emails to
address it all!
So, first things first: I just made a commit that will break everyone's build until they
do some updating.
The short story is:
(a) do an SVN update.
(b) add the property ant.home to your <user.home>/.hackystat/hackystat.site.properties
file.
Example: ant.home=c:\\apache-ant-1.6.5
(c) invoke 'ant -f autoconfig.build.xml' to regenerate a new modules.build.xml file with
the corrected
module dependencies for hackyApp_Pri (it needs hackySdt_Commit).
Mike: please update the daily build environment with ant.home before tonight's build so
it doesn't break due to this change.
The long story is:
One of the problems Aaron identified was the fact that the ant/lib is present in our
compilation environment. This has lots of negative implications due to the fact that the
sensor code is thus available during compilation. One of the negative implications is
hiding module dependencies, which Aaron discovered, and another is build failures if our
API were to change in certain ways due to circularities created by these jar files.
Aaron suggested setting fork="true" in the <javac> task to solve this, and that would
indeed solve the problem, but at the expense of a slowed down build. A solution that
doesn't slow down the build is to set includeAntRuntime="false" in the <javac> task.
(This attribute is optional and defaults to true, which is what created this problem in
the first place.)
So, that's what I did. This created two kinds of problems:
(1) hackyApp_Pri no longer compiled because it needed hackySdt_Commit. That was simple
to solve: I just added hackySdt_Commit to the list of dependent modules in
hackyApp_Pri/local.build.xml, and re-ran autoconfig to generate the correct dependency
structure.
(2) All hackySensor modules with an Ant interface no longer compiled because they needed
ant.jar. I solved this by first introducing a new requirement into the build: everyone
must now define ant.home in their hackystat.site.properties file. Given this, I could
update all of the hackySensor local.build.xml files to put ${ant.home}/lib/ant.jar into
the classpath.ref property.
Prior to committing the changes, I did a 'freshStart all.junit' and all the tests (except
hackyApp_TelemetryControlCenter, which was busted last night) passed.
I will address Aaron's other comments in a subsequent email. :-)
Cheers,
Philip
--On Monday, November 28, 2005 11:21 PM -1000 Aaron Kagawa <[EMAIL PROTECTED]>
wrote:
Hey Guys,
I've discovered some major problems with the Daily Diary and Daily Analysis
implementations.
--------------------------------------------
I've discovered a bug in hackySdt_FileMetric, while building the following
modules.
hackyCore_Build.available=true
hackyCore_Kernel.available=true
hackyCore_Installer.available=true
hackyCore_Common.available=true
hackyCore_Report.available=true
hackyCore_Statistics.available=true
hackyCore_Telemetry.available=true
hackySdt_FileMetric.available=true
I get these junit errors:
The class org.hackystat.sdt.filemetric.dailyanalysis.MostActiveFileMetric can
not be
found or the parameter is not correct
The class org.hackystat.sdt.filemetric.dailyanalysis.ProjectDailyAnalysis can
not be
found or the parameter is not correct
The class org.hackystat.sdt.filemetric.dailyanalysis.MostActiveFileWorkspace
can not be
found or the parameter is not correct
Note that: There seems to be no MostActiveFileMetric and MostActiveFileWorkspace
dailyanalysis.<name>.xml description file in the
org.hackystat.sdt.filemetric.dailyanalysis package. In addition, it seems that
the
Daily Diary doesn't contain these columns.
Also, note that there seems to be no MostActiveFile column either, which shows
that we
don't have a HttpUnit test the Daily Diary.
Checkout the ant output:
[echo] Warning: modules.build.xml file may be out of date. Regenerate
with: 'ant
-f autoconfig.build.xml'
[echo] (20:53:33) Completed hackyCore_Kernel.junit.
[echo] (20:53:35) Completed hackyCore_Statistics.junit.
[echo] (20:53:46) Completed hackyCore_Report.junit.
[echo] (20:53:54) Completed hackyCore_Common.junit.
[echo] (20:54:03) Completed hackyCore_Telemetry.junit.
[junit] Tests FAILED
[echo] (20:54:12) Completed hackySdt_FileMetric.junit.
[echo] (20:54:24) Completed hackyCore_Installer.junit.
[echo] (20:54:26) Completed all.junitReport
[echo] (20:54:26) Completed all.junit
Notice that, hackyCore_Installer.junit runs after hackySdt_FileMetric.. That
seems a
little strange to me. Note that I did run the ant -f autoconfig.build.xml.
So, I built and tested with all the SensorDataType modules and this is the Ant
output:
[echo] Warning: modules.build.xml file may be out of date. Regenerate
with: 'ant
-f autoconfig.build.xml'
[echo] (21:11:03) Completed hackyCore_Kernel.junit.
[echo] (21:11:05) Completed hackyCore_Statistics.junit.
[echo] (21:11:19) Completed hackyCore_Report.junit.
[echo] (21:11:33) Completed hackyCore_Common.junit.
[echo] (21:11:50) Completed hackyCore_Telemetry.junit.
[echo] (21:12:12) Completed hackySdt_Activity.junit.
[echo] (21:12:16) Completed hackySdt_Build.junit.
[echo] (21:12:20) Completed hackySdt_FileMetric.junit.
[echo] (21:12:29) Completed hackySdt_WorkspaceMap.junit.
[echo] (21:12:33) Completed hackySdt_Coverage.junit.
[echo] (21:12:42) Completed hackySdt_Dependency.junit.
[echo] (21:12:51) Completed hackySdt_Issue.junit.
[echo] (21:12:55) Completed hackySdt_ReviewActivity.junit.
[echo] (21:12:59) Completed hackySdt_ReviewIssue.junit.
[echo] (21:13:03) Completed hackySdt_UnitTest.junit.
[echo] (21:13:14) Completed hackyCore_Installer.junit.
[echo] (21:13:18) Completed hackySdt_CodeIssue.junit.
[echo] (21:13:22) Completed hackySdt_Commit.junit.
[echo] (21:13:26) Completed hackySdt_Perf.junit.
[echo] (21:13:29) Completed hackySdt_Cli.junit.
[echo] (21:13:35) Completed hackySdt_BuffTrans.junit.
[echo] (21:13:43) Completed hackySdt_BadData.junit.
[echo] (21:13:43) Completed all.junitReport
[echo] (21:13:43) Completed all.junit
Notice that the hackyCore_Installer.junit got pushed further down. AND that
there is
no junit errors anymore. Strange. So, what is going on?
So, I took a guess and just built and tested with the Activity and FileMetric
modules
and this is the Ant output:
[echo] Warning: modules.build.xml file may be out of date. Regenerate
with: 'ant
-f autoconfig.build.xml'
[echo] (21:42:27) Completed hackyCore_Kernel.junit.
[echo] (21:42:33) Completed hackyCore_Statistics.junit.
[echo] (21:42:46) Completed hackyCore_Report.junit.
[echo] (21:42:55) Completed hackyCore_Common.junit.
[echo] (21:43:05) Completed hackyCore_Telemetry.junit.
[echo] (21:43:29) Completed hackySdt_Activity.junit.
[echo] (21:43:36) Completed hackySdt_FileMetric.junit.
[echo] (21:43:47) Completed hackyCore_Installer.junit.
[echo] (21:43:47) Completed all.junitReport
[echo] (21:43:47) Completed all.junit
So, it seems that there is something about the hackySdt_Activity module that
makes
hackySdt_FileMetric work correctly.
Ahh.... I just figured it out... In Eclipse we set up the .project files such
that
hackySdt_FileMetric depends on hackySdt_Activity for the MostActiveFileMetric
dailyanalysis implementation. But, the Ant hackySdt_FileMetric.required.modules
property isn't correct.
But, the strange thing is that our build compiles even though, I didn't build
hackySdt_Activity but, hackySdt_FileMetric uses the
activity.dailyanalysis.MostActiveFile class. Say it with me...... Ahh.... That
is
strange....
The problem took some digging, but I figured it out:
c:\java\svn\hackyCore_Build>ant -verbose hackySdt_FileMetric.compile
....
[hackyCore_Build.javac] Using modern compiler
[hackyCore_Build.javac] Compilation arguments:
[hackyCore_Build.javac] '-deprecation'
[hackyCore_Build.javac] '-d'
[hackyCore_Build.javac] 'C:\java\svn\hackyCore_Build\build\war\WEB-INF\classes'
[hackyCore_Build.javac] '-classpath'
[hackyCore_Build.javac] 'C:\java\svn\hackyCore_Build\build\war\WEB-INF\classes;C
....
va\apache-ant-1.6.5\lib\locc.jar;C:\java\apache-ant-1.6.5\lib\sensor.ant.jar;C:\
java\apache-ant-1.6.5\lib\sensor.checkstyle.jar;C:\java\apache-ant-1.6.5\lib\sen
sor.junit.jar;C:\java\apache-ant-1.6.5\lib\sensorshell.jar;C:\java\apache-ant-1.
6.5\lib\xercesImpl.jar;C:\java\apache-ant-1.6.5\lib\xml-apis.jar;C:\java\jdk1.5.
0_04\lib\tools.jar'
[hackyCore_Build.javac] '-sourcepath'
[hackyCore_Build.javac] 'C:\java\svn\hackySdt_FileMetric\src'
[hackyCore_Build.javac] '-target'
[hackyCore_Build.javac] '1.4'
[hackyCore_Build.javac] '-g'
[hackyCore_Build.javac] '-source'
[hackyCore_Build.javac] '1.4'
[hackyCore_Build.javac]
[hackyCore_Build.javac] The ' characters around the executable and arguments are
[hackyCore_Build.javac] not part of the command.
[hackyCore_Build.javac] Files to be compiled:
[hackyCore_Build.javac] C:\java\svn\hackySdt_FileMetric\src\org\hackystat\sd
t\filemetric\dailyanalysis\MostActiveFileMetric.java
You should see the problem. AHHH!!! So, it seems that the javac task in the
build.xml
should use "fork". To spawn a new classpath instead of appending to Ant's
classpath.
---------------------------------------------
I disagree with putting TestDailyDiary and TestProjectManager in the
org.hackystat.std.activity.dailyanalysis package. Instead, each dailyanalysis
package
should have a TestDailyDiary<analysis name> class that tests their portion of
the daily
diary. If we had a HttpUnit test for each of the DailyAnalysis implementations
then we
would find the problems with the XML description files.
Also, it is really, really, really strange to find TestProjectManager in the
activity.dailyanalysis package. I think I would feel a little better if the
package
was named differently. First of all, it has nothing to do with the
dailyanalysis
package. So, why did we even put in that specific package. What makes the
dailyanalysis package better than the reducer package? Would it be so bad to
put it in
hackySdt_Activity/src/org/hackystat/core/common/project?
---------------------------------------------
The dailyanalysis.basic.xml in the org.hackystat.core.common.dailyanalysis
contains the
wrong class name:
<dailyanalysis>
<!-- in dailydiary package -->
<analysis name="TimeStamp"
class="org.hackystat.stdext.dailydiary.dailyanalysis.FiveMinuteTimeStamp"
enable="false"/>
</dailyanalysis>
Another strange find.... This file should be in the
org.hackystat.core.common.dailydiary.dailyanalysis package, because that is
where the
FiveMinuteTimeStamp implementation lives. And of course the class name should be
changed. In addition, if we had a TestDailyDiary<analysis name>.java HttpUnit
test
case, then we would have found that.
Its also interesting to note that 100% coverage would have not found this error,
because the DailyDiary uses reflection. In addition, we have client side tests
of the
individual Daily Analysis code. Thus, if we tested one Daily Analysis
implementation
through the web interface and tested it through the client side API, then we
would have
100% coverage. So.... not sure how to do a quality assurance check on this
type of
situation.
---------------------------------------------
Action Items:
1) Add an Ant dependency on hackySdt_Activity in the
hackySdt_FileMetric/build.xml
2) Change the javac task to use fork in the hackyCore_Build/build.xml
3) Change the class name in
org.hackystat.core.common.dailyanalysis/dailyanlaysis.basic.xml file.
4) add MostActiveFile, MostActiveFileMetric, MostActiveFileWorkspace to a
dailyanalysis.<name>.xml file.
5) Add Test Cases in the dailyanalysis packages, that actually tests the Web
Interface.
thanks, aaron
ps. I was thinking that it would be cool if my sensor data would indicate that
I was
debugging a problem. I have little to no active time, but I sent a bunch of
Build,
Unit Test, and Activity data (a lot of code reading and browsing). I spent a
couple of
hours tonight figuring all this out and wanted to know how much time I spent in
the
past figuring this kind of stuff out. Any ideas?? Maybe Hongbing's micro
development
streams can discover "debugging".
By the way, I rediscovered Eudora's Statistics function (btw, Eudora is an email
client) and found that I am projected to spend 143.4 hours writing emails like
these.
On the other hand, I'm projected to spend 30.9 hours reading emails. That's
really
cool that Eudora knows what is reading versus composing, that it has "Active
Time", and
that it can project my yearly trends.
So, It would be cool if Eclipse had a mode where a Java file is initially not
editable,
then easily changed with a key stroke or button to make it editable. Then we
should be
able to know Reading versus Composing.
