Larry wrote: " The fence is not a general selection solution. It is a very specialized one. With my new Extract Layer in Fence plugin, I can create smaller subsets of the million point layer in seconds. Selection will not do this at all."
Understood. You are building a special tool for a special circumstance. There is nothing wrong with that as long as we keep it as a plug-in. I only spoke up before because I thought we were talking about integrating this into the core as a major "selection alternative". Thank you for clearing up my misconception. Larry wrote: "As to which use cases are "normal "and which are "extreme", this is a moving target. Personally, I am happy when I see people stretching the limits of JUMP. Nothing makes inefficient code apparent like extreme cases." This is a perspective that I hadn't considered. There is no arguing that you are the best OpenJUMP programmer when it comes to identifying and fixing performance bottlenecks, at least in my opinion. Your fixes have been great! I still think there is a balance to be maintained. Every performance fix comes with a cost in manpower and refactoring. We should remember to ask "is this really worth it" before those changes are made. I guess this is the old cost-benefit argument. I do trust your judgement in this regard Larry. If you have suggestions on how to improve the performance of the existing selection code I'd like to hear about them. Maybe I can visit this area of the code again in the future. The Sunburned Surveyor On 10/8/07, Sunburned Surveyor <[EMAIL PROTECTED]> wrote: > Michael, > > I can offer a tiny bit of insight into OpenJUMP's selection code. What > I'm about to tell you is part of what I hope to document about the > selection system when I'm done squishing bugs on my Super Select Tool. > > OpenJUMP uses collections and something called a CollectionMap quite > extensively in the selection code. (A CollectionMap is a utility class > used by vivid Solutions in JUMP. It is a Map whose values are other > Collections.) > > At a bare minimum OpenJUMP maintains a single CollecctionMap for each > layer that contains selected items. The keys of this CollectionMap are > the selected Features and the values are another Collection. The > Collections stored as values are LinkedLists (I believe) and they > contain the "items" of the selected Features. In most cases this is > just the JTS Geometry object representing the selected Feature, but in > some cases it might be the "parts" of a JTS Geometry. (A sement of a > polygon ring, for example.) > > As far as I can tell the CollectionMap is not deleted once it is > created (as long as the instance of OpenJUMP is running) although it > may be "empty" if no features in the correspongind layer are selected. > > If I remember correctly several other collections are created in the > process of "drilling down" to the correct "items" to select based on > user actions. > > I know that I am a rookie code slinger, but I found that the selection > code in OpenJUMP was a real challenge to understand. I hope to > supplement/add some Javadoc comments that might make it easier on the > next programmer to venture that direction. > > I hope my tiny bit of info about the selection code will shed some > light on the topic being discussed currently. > > The Sunburned Surveyor > > > On 10/8/07, Michaël Michaud <[EMAIL PROTECTED]> wrote: > > Hi Larry, > > > > May be I did not understand your solution, but I think like SS, that > > selecting with a fence is not a general solution for the selection > > problem. A selection can be the result of a query or any set of features > > without any particular spatial relationship. > > I wonder why collections containing selection are so "huge". Basically, > > a selection should be a set of handles (or a little more to be able to > > manage geometry parts). > > In the case of a large point layer without attribute, the selection size > > may be comparable to the feature layer size, but it should stay smaller, > > and it is really a worst case. > > Maybe there are other reasons why the selection is so large and slow : > > hard copy of features, cache, undo/redo management ? > > > > Anyway, thanks for the continuous improvement of OpenJUMP performances. > > > > Michaël > > > > Larry Becker a écrit : > > > > > Hi SS, > > > > > > I agree that the problem with selection is the large collections > > > being generated. "Parts" are only part of the problem and it is a bit > > > late to stop supporting this capability. Optimizations are always > > > possible, but only if you have a clear understanding of what you are > > > optimizing and use benchmarks for metrics. > > > > > > Larry > > > > > > On 10/8/07, *Sunburned Surveyor* <[EMAIL PROTECTED] > > > <mailto:[EMAIL PROTECTED]>> wrote: > > > > > > This sounds like a great solution Larry, but I was wondering if you > > > might be able to confirm why selection requires more memory than a > > > fence. > > > > > > Is it because when you use "selection" a bunch of Java collections > > > are > > > created, whereas when you use a fence, you are just creating a > > > geometry? Is it because with the fence no data structures for the > > > features within the fence are created until you actually want to do > > > something? > > > > > > The only problem I see with using a fence is that it wouldn't allow > > > complex selections. (For example, if I use a fence all the features > > > are the fence are in the "quasi-selection".I can't go in an remove > > > some items from the quasi-selection. Does this make sense?) > > > > > > I worked with some of the selection code recently, and it seemed very > > > complex to me. There were so many collections being used in the code I > > > had to write the purpose of each one down on a scratch pad so I could > > > keep track. It seems that most of this complexity was a result of > > > OpenJUMP's ability to select parts of Features, and not being > > > restricted to selection of whole features. > > > > > > I wonder if we couldn't come up with a much simpler selection system > > > that only selected whole features. We could probably do this using a > > > single collection that stored only the FID of a Feature, or an object > > > reference to the Feature. A collection of a couple million FID values > > > would only be a couple of MB correct? > > > > > > Perhaps your idea with the fence is best, but I'm thinking it might be > > > better to address the complexity in the selection system itself. > > > > > > I wonder how much people are using OpenJUMP's ability to select > > > "parts" of features. > > > > > > Just my thoughts. I don't think it would take me long to cook up the > > > simple "whole-feature-selection-only" system I am talking about. > > > > > > The Sunburned Surveyor > > > > > > On 10/8/07, Stefan Steiniger <[EMAIL PROTECTED] > > > <mailto:[EMAIL PROTECTED]>> wrote: > > > > simple solution and well thought :) > > > > > > > > we should go for > > > > > > > > stefan > > > > > > > > Larry Becker wrote: > > > > > > > > > Hi, > > > > > > > > > > Recent work with Extract Layers by Geometry Type, and improving > > > > > OpenJump's support for very large selections, has convinced me > > > that > > > > > JUMP needs a mechanism for dealing with subsets of large layers. > > > > > Although the selection mechanism is very flexible, it is very > > > > > expensive in terms of memory and processing. To seem what I mean, > > > > > consider Arnd's million point layer problem. To break the > > > layer into > > > > > more manageable pieces, he tried to select some of the points > > > and move > > > > > them to another layer. When this is done with such a large > > > selection, > > > > > the UI must build gigantic data structures. The result is an > > > > > extremely slow response or an out of memory error. > > > > > > > > > > An approach that might work for some problems is to use the > > > Fence tool > > > > > to indicate a subset of the data that can be used in place of a > > > > > selection. The advantage is that no data structures are > > > needed and > > > > > there are no selection handles to draw. I was thinking of > > > producing > > > > > another Layer Extract plugin that would extract features in > > > the Fence > > > > > to a new layer. By coding support methods in a utility class, we > > > > > could gradually add support for "features in fence" subsets to the > > > > > other tools. > > > > > > > > > > What do you think? > > > > > > > > > > regards, > > > > > > > > > > Larry Becker > > > > > > > > > > -- > > > > > http://amusingprogrammer.blogspot.com/ > > > > > > > > > > > > > > > >------------------------------------------------------------------------ > > > > > > > > > > > > > > > >------------------------------------------------------------------------- > > > > >This SF.net email is sponsored by: Splunk Inc. > > > > >Still grepping through log files to find problems? Stop. > > > > >Now Search log events and configuration files using AJAX and a > > > browser. > > > > >Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > > > > > > > > > > > > > >------------------------------------------------------------------------ > > > > > > > > > >_______________________________________________ > > > > >Jump-pilot-devel mailing list > > > > >Jump-pilot-devel@lists.sourceforge.net > > > <mailto:Jump-pilot-devel@lists.sourceforge.net> > > > > >https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > <https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel> > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > This SF.net email is sponsored by: Splunk Inc. > > > > Still grepping through log files to find problems? Stop. > > > > Now Search log events and configuration files using AJAX and a > > > browser. > > > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > > _______________________________________________ > > > > Jump-pilot-devel mailing list > > > > Jump-pilot-devel@lists.sourceforge.net > > > <mailto:Jump-pilot-devel@lists.sourceforge.net> > > > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > This SF.net email is sponsored by: Splunk Inc. > > > Still grepping through log files to find problems? Stop. > > > Now Search log events and configuration files using AJAX and a > > > browser. > > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > _______________________________________________ > > > Jump-pilot-devel mailing list > > > Jump-pilot-devel@lists.sourceforge.net > > > <mailto:Jump-pilot-devel@lists.sourceforge.net> > > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > <https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel> > > > > > > > > > > > > > > > -- > > > http://amusingprogrammer.blogspot.com/ > > > > > >------------------------------------------------------------------------ > > > > > >------------------------------------------------------------------------- > > >This SF.net email is sponsored by: Splunk Inc. > > >Still grepping through log files to find problems? Stop. > > >Now Search log events and configuration files using AJAX and a browser. > > >Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > > > >------------------------------------------------------------------------ > > > > > >_______________________________________________ > > >Jump-pilot-devel mailing list > > >Jump-pilot-devel@lists.sourceforge.net > > >https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > > > > > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > Jump-pilot-devel mailing list > > Jump-pilot-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
