Leif Mortenson wrote: > Alternate Profiler Proposal. > > I spent all last week working through the issues of profiling and came up with an API > which is quite a bit different that the current API in the profile package. > > I have checked in a new API in the altprofile package in the scratchpad.
Excellent! > I had the goal of coming up with a system which would make it easy to design > Profilability into any component (Note lower case ‘c’) without having >to worry > about performance. This is the same thinking that has gone into many logging systems > of late. You add lots of debug logging to classes to make it easier to debug them at > a later date. This profiling system follows the same line of thinking. Ok. My approach was to send a startProfiling() message to the Profilables being monitored, and then send an endProfiling() message when it is all done. However, I am excited to have a working tool available. > A component can be made profiler ready by implementing the Profilable interface. > This interface allows a component to expose a number of ProfilePoints as well as > tell the Profiler about any child Profilables. So far we are in sync. > ProfilePoints are extremely light weight when they are not connected to the Profiler. > So components can add ProfilePoints without having to worry about a performance hit > when the Profiler is not in use. Yep. Mine too. The biggest difference I believe is the relationship of ProfileReport and Profiler. > This implementation has been divided into 4 packages: > > 1) excalibur.altprofile: Defines the base interfaces and classes that most components > will directly make use of. Excellent. > 2) excalibur.altprofile.component: Defines a new ProfilerComponentManager which >extends > the ExcaliburComponentManager and understands how to deal with Profilable components. Kool. I will most likely throw in something like this in the Container/ ContainerManager stuff as well. I will be refactoring a few things to make that code easier to use though... > 3) excalibur.altprofile.profiler: This package contains the guts of the system. It > defines a ProfilerManager which is where all the work of profiling an Avalon > application is handled. Excellent. > 4) excalibur.altprofile.profiler.gui: This package contains a reference >implementation > of a simple GUI which enables you to browse and view the profiler aware elements of > an application. "We're not worthy! We're not worthy!" Great work guys. I mean it. > The excalibur.altprofile.component package required that I make a couple changes to > the excalibur.component package to allow key classes to be extended. That's fine. > So far, I have temporarily made the ResourceLimitingJdbcDataSource in the scratchpad > Profilable to make it easy to test things out. The ProfileManager also includes > default profile points for monitoring the JVM memory. Excellent. > Lets start out with a sample piece of code that will start up an application using > the ProfilerComponentManager and a Profiler GUI. This code should be familiar to > anyone who has worked with the Excalibur ComponentManager: > > ------ > DefaultContext context = new DefaultContext(); > > context.put("root", getServletContext().getRealPath("/")); > context.makeReadOnly(); > > DefaultConfigurationBuilder builder = new >DefaultConfigurationBuilder(); > Configuration systemConfig = builder.build(configStream); > > DefaultLogKitManager logManager = new DefaultLogKitManager(); > Logger lmLogger = >Hierarchy.getDefaultHierarchy().getLoggerFor("log-mgr"); > >lmLogger.setPriority(Priority.getPriorityForName(systemConfig.getAttribute("log", >"INFO"))); > logManager.setLogger(lmLogger); > logManager.configure(systemConfig.getChild("logkit")); > > > // Set up the Profiler Manager > DefaultProfilerManager profilerManager = new DefaultProfilerManager(); > > profilerManager.enableLogging(new >LogKitLogger(logManager.getLogger("pm"))); > profilerManager.configure(systemConfig.getChild("profiler")); > profilerManager.initialize(); > > // Set up the Role Manager > Configuration roleConfig = builder.build(roleStream); > DefaultRoleManager roles = new DefaultRoleManager(); > > roles.setLogger(logManager.getLogger("rm")); > roles.configure(roleConfig); > > > // Set up the Component Manager > ProfilerComponentManager manager = new ProfilerComponentManager(); > > manager.setLogger(logManager.getLogger("cm")); > manager.setLogKitManager(logManager); > manager.contextualize(context); > manager.setProfilerManager(profilerManager); > manager.setRoleManager(roles); > manager.configure(systemConfig.getChild("environment")); > manager.initialize(); > > // Set up the ProfilerFrame > ProfilerFrame profilerFrame = new ProfilerFrame( profilerManager, >"Nikko System Profiler" ); > profilerFrame.setVisible( true ); > ------ > > The differences are that this code creates a ProfilerManager and assigns it to the > ProfilerComponentManager using the setProfilerManager() method. The code then > continues to create a ProfilerFrame passing the ProfilerManager to the constructor. > The ProfilerFrame is not required by the Profiler. It is just a default way to view > the Profile data. By the way, It was suggested to me a while back that we should rename the Profiler stuff as Instrumentor, InstrumentationPoint, etc. The reason being that we are not truly profiling, but instrumenting the system. I would like to get this code up and running and final as soon as possible. (I have a book to write, and this can really make an impact). > Without any other changes, your application should be able to be run using the above > code to launch the application. But at this point none of the Profilable components > will be profiled because they have not been configured in the config file yet. > > Components which implement Profilable are not added to the list of components to be > profiled by default. To make them profilable, you must add a >‘profilable’ > attribute which gives them a profiler name. > > Here is an example datasource configuration: > ------ > <datasources> > <jdbc name="asp-service" profilable="jdbc-asp-service"> > <pool-controller min="1" max="10"/> > <auto-commit>true</auto-commit> > <driver>org.postgresql.Driver</driver> > <dburl>jdbc:postgresql://host/asp_service</dburl> > <user>user</user> > <password>pwd</password> > </jdbc> > </datasources> > ------ Ok. Looks good. We *may* want to make that alterable during run-time though. For instance, the GUI may want to select specific Profilables that we are monitoring at any one time. > Assuming the roles is configured to use a ResourceLimitingJdbcDataSource for the >above > datasource, the profiler will now recognize the DataSource as a Profilable named > 'jdbc-asp-service'. > > An application can now request to be added as a listener of update events for any one > of 3 ProflilePoints published by the DataSource. > > To make things work really nicely though, we also need to configure the Profilable > using the profiler configuration. It takes on the following format: > > ------ > <profiler> > <profilables> > <profilable name="profiler" > description="Profiler"> > <profile-point name="total-memory" > description="Total Memory"> > <sample type="max" interval="1000" size="600" > description="Maximum each second."/> > <sample type="max" interval="60000" size="1440" > description="Maximum each minute."/> > <sample type="max" interval="3600000" size="720" > description="Maximum each hour."/> > </profile-point> > > <profile-point name="free-memory" > description="Free Memory"> > <sample type="min" interval="1000" size="600" > description="Minimum each second."/> > <sample type="min" interval="60000" size="1440" > description="Minimum each minute."/> > <sample type="min" interval="3600000" size="720" > description="Minimum each hour."/> > </profile-point> > > <profile-point name="memory" > description="In-Use Memory"> > <sample type="max" interval="1000" size="600" > description="Maximum each second."/> > <sample type="max" interval="60000" size="1440" > description="Maximum each minute."/> > <sample type="max" interval="3600000" size="720" > description="Maximum each hour."/> > </profile-point> > </profilable> > > <profilable name="jdbc-asp-service" > description="ASP Service JDBC Pool"> > <profile-point name="pool.new-poolables" > description="New Connections"> > <sample type="ctr" interval="1000" size="600" > description="Count / second."/> > <sample type="ctr" interval="60000" size="1440" > description="Count / minute."/> > <sample type="ctr" interval="3600000" size="720" > description="Count / hour."/> > </profile-point> > > <profile-point name="pool.size" > description="Open Connections"> > <sample type="max" interval="1000" size="600" > description="Maximum each second."/> > <sample type="max" interval="60000" size="1440" > description="Maximum each minute."/> > <sample type="max" interval="3600000" size="720" > description="Maximum each hour."/> > </profile-point> > > <profile-point name="pool.ready-size" > description="Available Connections"> > <sample type="max" interval="1000" size="600" > description="Maximum available each >second."/> > <sample type="max" interval="60000" size="1440" > description="Maximum available each >minute."/> > <sample type="max" interval="3600000" size="720" > description="Maximum available each >hour."/> > > <sample type="min" interval="1000" size="600" > description="Minimum available each >second."/> > <sample type="min" interval="300000" size="288" > description="Minimum available each >minute."/> > <sample type="min" interval="3600000" size="720" > description="Minimum available each >hour."/> > </profile-point> > </profilable> > </profilables> > </profiler> > ------ I'm kind of luke-warm on this. We should be able to determine the type of sample we are accepting automatically. The reporting agent (whether gui or file) should be able to adjust the report to reflect the per second, per minute, per hour differences. Especially since it is controlling the sampling frequency directly. > This example profiler configure configures two Profilables. The first called >"profiler" > configures the ProfilerManager. The second, “jdbc-asp-service” >configures our example > DataSource. > > By default, ProfilePoints are purely pull based profiling objects. In order to make >the > profiling data available for a user to see at any time, the data must be collected >and > stored into history. This is done by defining one or more Samples for each Profile >Point. > > The system comes with two types of ProfilePoints. The first, CounterProfilePoint, >can be > used to profile the number of times an action takes place. It exposes an increment() > method. The second, ValueProfilePoint, is used for profiling absolute values. >Things > like pool sizes, etc. It exposes a setValue(int value) method. > > CounterProfilePoints can be assigned samples of type “counter” >(“ctr”). > ValueProfilePoints can be assigned samples of types “maximum” >(“max”), > “minimum” (“min”), and “average” >(“avg”). > > Taking the “maximum” types sample as an example. It will monitor each >call to setValue > for the ProfilePoint and store the maximum value in a particular sample period as the > value. “minimum” stores the minimum value, and >“average”, the average value for the > sample period. The “counter” sample will store the number of times >that increment() > was called on the ProfilePoint during the sample period. > > The ProfilePoints which have samples visible will be viewable as graphs in real time > using the ProfilerFrame class. > > Right now, this shouldn’t be too difficult to get set up. But I get a example > application that will work out of the box very soon. > > Wanted to get something up as soon as possible though because it looks like Marcus > has also been giving this a lot of thought. > > Cheers, I am sure you will have some questions about how things work. It is late > here now. I should be able to answer them in the AM. So far, kudos for the excellent work. I will review the interfaces shortly, and let you know if I have any questions. -- "They that give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>