It would be great if someone could test that the resolver still passes the OSGi CTs. The Felix standalone resolver is not yet used in the Felix framework (FELIX-4525) but it is used by Equinox, so ideally someone could check the Core Framework CT with Equinox and this updated resolver to make sure that it still passes all the tests there.
On 30 September 2014 09:44, Guillaume Nodet <[email protected]> wrote: > I recently run into OutOfMemory problems when using the resolver in > addition to the slowness I already raised. > I worked a bit on the resolver and pushed my results to a private branch > for review: > https://github.com/gnodet/felix/commits/resolver-improvements > > The memory problem was addressed by replacing the internal data structures > hold by the Candidates class. > This class was using a HashMap to store the Map<Requirement, > List<Capability>> for candidates and for each permutation, the whole data > structure was copied. > Unfortunately, the HashMap object is very memory intensive and the > List<Capability> were duplicated a lot during resolutions. > Therefore, I introduced a few collections which are optimized for memory > consumption. Those are the OpenHashMap (derived from the mahout > collections) and the CopyOnWriteList/CopyOnWriteSet. > The OpenHashMap uses open adressing and two Object[] internally instead of > tons of entry objects. Copy should also be faster as much less objects are > created. The values in this map are now CopyOnWriteList instead of > ArrayList, which has the big benefit of not duplicating the entire arrays > when the structure is copied. Those two collections work roughly the same > way as CopyOnWriteArrayList but without any thread safety. In addition, > creating a new list from a CopyOnWriteList does not lead to creating a new > Object[] to store the data, but merely assign the same pointer internally. > In terms of memory consumption, this means overall, that copying a > Candidates class, will lead to two Object[] arrays creation (for the > OpenHashMap) and the creation of small objects for the lists (but with no > copy of the data itself). > This is mainly commit > https://github.com/gnodet/felix/commit/0bf1523f21f9983b21b2737b4f78bb8d78cd35fd > > The slowness problem was partially addressed because I found out that the > resolver was attempting the same resolution multiple times. I think this > is due to the order or removing possible candidates and multiple paths to > the same resolution were executed. In order to solve this problem, the > Candidates now holds a m_path structure which contains all the removed > candidates. This object is used in the resolver loop to make sure we > haven't already tried the same resolution. For big resolutions, I think it > will improve things a lot. > This is fixed by commit > https://github.com/gnodet/felix/commit/090a67a7fc05170291ad9cff808229a0292b6fb2 > > > I'm going to do further testing, and I'm going to add a big resolution test > to measure performances and memory consumption. > In the mean time, I'd like others to review and test it if possible. > > > 2014-09-15 16:23 GMT+02:00 Guillaume Nodet <[email protected]>: > >> There are a bunch of bug fixes and enhancements in the resolver that >> haven't been released yet. Is there anything pending on resolver 1.2.0 or >> is it ready to be released ? >> >> See >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310100&version=12326783 >> >> Cheers, >> Guillaume Nodet >> >> >>
