I agree, any attempt at improvement wouldn't be general. thanks for the explanation. On Thu, Dec 6, 2018 at 10:45 AM Dawid Weiss <dawid.we...@gmail.com> wrote: > > I don't think it makes much sense, to be honest. Without actual > reflection you're binding things to a particular implementation > (you're assuming this and that JDK implementation). That's why we > decided to remove it instead of making it overly complex (and possibly > untrue). If a test is using it, perhaps the test itself should be > fixed. > > D. > On Thu, Dec 6, 2018 at 4:35 PM Michael Sokolov <msoko...@gmail.com> wrote: > > > > That's what it looked like to me, too. I wonder if it would be worth > > improving the estimate for some very common Collections classes? I see > > this comment eg in BaseIndexFileFormatTestCase: > > > > // we have no way to estimate the size of these things in codecs > > although > > // something like a Collections.newSetFromMap(new HashMap<>()) uses > > quite > > // some memory... So for now the test ignores the overhead of such > > // collections but can we do better? > > > > This is in testRamBytesUsed and there is a kind of fudge factor in > > there for handling mismeasurement errors of the sort we are talking > > about. Actually the test seems to be more about validating the > > RamUsageTester than about validating the accounting in SegmentReader! > > There are lots of other usages in tests, but I suppose they don't > > require very precise handling of Collections classes (since they > > pass)? Anyway it is certainly possible to improve the estimate quite a > > bit and pretty easily for HashMap by simply counting the size of the > > Node that is used for each entry, although given the dynamic nature of > > these data structures (HashMap eg can use TreeNodes sometimes > > depending on data distribution) it would be almost impossible to be > > 100% accurate. > > On Thu, Dec 6, 2018 at 7:14 AM Dawid Weiss <dawid.we...@gmail.com> wrote: > > > > > > > It's entirely possible it fails to dig into Maps correctly with newer > > > > Java > > > > releases; maybe Dawid or Uwe would know? > > > > > > We have removed all reflection from that class a while ago exactly > > > because of encapsulation issues introduced in newer Java versions. > > > > > > https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/util/RamUsageEstimator.java > > > > > > I think you may be thinking of RamUsageTester which is in the test > > > framework and indeed accumulates only keys and values from iterables. > > > These methods are for tests only and are rough. You shouldn't rely on > > > them for accurate memory consumption accounting (instead, use the > > > Accountable interface). > > > > > > Dawid > > > > > > > > > D. > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org >
--------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
