Adrian Custer wrote: > single instance of a CRS Authority (FactoryOnOracleSQL, for > example), *we can expect* the code to slow to a crawl... > You are correct - mostly my text was just to provide context. The bulk of my effort has gone into working with my customer to understand there needs. My summary is perhaps to glib :-(
If I was more blunt; the referencing module does not look like normal Java EE code and our customer is paying us to clean it up. That sounds simple, but the details are important. The specific needs are serious and most of the show stoppers are around the handling of connections. Trust in the concurrency model is also an issue, it needed to be thought through and verified with sequence diagrams. It is more a matter of not wasting resources (starting too many threads that are each holding a connection) than anything to do with performance. So please don't pick apart my text - the only reason this is one proposal rather than three is the fact that the problems are all tangled up together. > Jody, would it be possible for you to explain better why you think this > is such critical work? Is it possible to create a simple stress testing > application which models the situation you envision and shows the > referencing slowing to a crawl? Since your work is essentially an > optimisation effort, perhaps such a set of stress tests and benchmarks > would be warranted regardless? > Well I can do slightly better - I am looking at a fork of the entire referencing module created in order to patch up the three problems being solved. The fact that our code cannot be used in its current state is enough reason to fix it... > As this work progresses, it seems to be placing serious strain on > Martin. In his usual understated way, he merely emitted a quiet request > that the work be delayed for one release. From my point of view, a > better process is needed for the integration of this code. > I also agree that the work does not need to be placed into GeoTools 2.4. It just needs to proceed - and it is proceeding. The only downside is my time is spent on trunk, so Justin and Martin can decide if they want to branch off 2.4 now (or later) based on my availability. I expect to see a bit more work put into GeoTools 2.4 before we fork, DataAccess needs to be removed, FilterVisitor needs to be accounted for, and so on. It looks like Justin is asking that the SimpleFeatureBuilder and SimpleFeatureTypeBuilder hook up be done later (even though that is not my preference). This is a resource/planning issue - hence my picture of what is going on. > For any change to the modules he maintains, Martin will require of > himself that he analyze and fully understand every single piece. His > review covers everything from the proper existence of svn history to > correct javadoc commenting to proper logic. > I understand; we are guests in the referencing module. And we asked martin to hold off his review until Friday. I think we are responding in code (cleaning up javadocs etc...) based on his feedback. The other way to ease the strain is to set a data for 2.4.x branch that is later next week. I best stop - I need to get back to the code side of things. I am sorry I missed the IRC meeting. > Unfortunately, the current changes are being done in a manner which is > straining Martin. He spent the weekend not just on review but also on > cleanup, for example, backing out files from svn only to re-commit them > with the history of their origin correctly integrated (svn copy not svn > add). Since Martin is under the gun his own projects, having to babysit > code he considers stable is causing strain. > I apologize for what was a mistake handling svn on my part. > Jody, perhaps we could look for a more systematic approach to > integrating this code. Surprisingly, this may even require such things > as us re-coordinating our shared strategy around our use of svn. > Similarly, it may be necessary to figure out a way to chunk the work > into bite size pieces and to find some way to schedule their integration > through Martin's review. > I don't have time the proposal process took a month; this leaves me crunched on the coding side. :-( > Finally, to clarify your constraints around this effort, perhaps you > could explain your scheduling constraints: is the work *due*, if so what > is the promised *product*, does this work block future work...? > My work is divided into two week sprints. This sprint I am implementing the proposal, next sprint I am testing and documenting (including javadocs). It holds up future work on (I hope) ISO Feature, Geocoding and so on. > Answers to those questions might help Martin prioritize some of his own > work before reviewing yours, which might reduce the stress on him. > Let's see if we can set a date for 2.4.x which would take the strain off both of us. > I hope this clarifies the non-technical issues around your work. > Hopefully, this can be sorted out quickly so everyone can get back to > this exciting summer of work. > Thanks Adrian, this geotools change proposal thing is a work in progress. Let's keep that in mind next time we vote to approve a proposal. The proposal does include a timing section (under tasks) - and we should make sure the time is correct and works nicely with other projects. It is the work of the PMC and we are not doing too bady; I would like to be less of a strain on Martin - at the same time I cannot go over the proposal contents again and again via email. I best start linking to the wiki pages and sequence diagrams rather than explaining things poorly a second time. As example the ObjectCache design and locking is motivated by the mediator roll played by ThreadedEpsgAuthority ... we need that guy to block on READ (ie the get method) so that more and more workers are not created. There is a narrow window of opportunity for multiple workers to be created ... hence the extra check after a write lock is acquired using the peek method. Cheers, Jody ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Geotools-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geotools-devel
