Thanks, Greg. I guess a branch is good, in case I'd like to suggest any tweaks.
-- Josh Tynjala Bowler Hat LLC https://bowlerhat.dev/ On Tue, May 12, 2026 at 10:10 PM Greg Dove <[email protected]> wrote: > Josh, I *think* it might be a combination of the two. I'm asking others who > were seeing it more often than I did to test a possible fix (I will share > an updated compiler build with them), because repro of the actual issue is > still quite challenging. > *if* it works (maybe will need 1-2 days to be sure), I'll push it either > directly to dev or via branch/PR (lmk what you prefer) and I'd certainly > appreciate your review of that if possible. I did use a bit of AI support > for the sleuthing and the testing, but you have spent much more time in the > compiler codebase than I have. > For now, though, fingers crossed.... > > > > On Wed, May 13, 2026 at 4:00 AM Josh Tynjala <[email protected]> > wrote: > > > Thanks for the update, Greg. Threading could certainly be a cause if we > > arre missing some kind of synchronization. I know that we have > > workspace.startBuilding() and workspace.startIdleState() as ways of > > ensuring threads are under control. We may be missing one of those calls > > somewhere before emitting JS. > > > > As for GC, I recall that reducing JVM memory wasn't necessarily enough > for > > me to reproduce the other GC related bug I mentioned, strange as that > > seems. I remember also adding System.gc() calls in various places > (though I > > don't remember exactly where), and I think that's what finally allowed me > > to reproduce the issue semi-reliably. > > > > -- > > Josh Tynjala > > Bowler Hat LLC > > https://bowlerhat.dev/ > > > > > > On Mon, May 11, 2026 at 10:15 PM Greg Dove <[email protected]> wrote: > > > > > Hey Josh, > > > > > > I am actively looking into this again. I am less convinced that it is > GC > > > related (I reduced memory allocation to low levels) and perhaps it is > > more > > > to do with threads/race-conditions. But it's very difficult to be > sure, I > > > spent today adding logging and trying to repro, but did not repro the > bug > > > all day. I will keep on this tomorrow trying to find the right > conditions > > > to force it to occur. If I can figure out what those are, I will share > > them > > > with you. > > > > > > -Greg > > > > > > > > > On Tue, May 5, 2026 at 9:06 AM Greg Dove <[email protected]> wrote: > > > > > > > Wake up brain (self talk): > > > > "and then not wrong for subsequent output" <- should be of course > "and > > > > then wrong for subsequent output". > > > > > > > > On Tue, May 5, 2026 at 9:05 AM Greg Dove <[email protected]> > wrote: > > > > > > > >> Thanks for looking into this, Josh. > > > >> > > > >> "If it isn't too difficult to reproduce" > > > >> Quick comments, just in case it helps: > > > >> > > > >> It was not something I could repro for debugging purposes in the > > > >> compiler. It was still 'rare' in practice - max 2-3 times per day > > that I > > > >> observed, sometimes only once a day - and not manifesting in the > same > > > code > > > >> - although perhaps that is simply because code can change a lot > > between > > > >> compiler runs - and "awareness" was based on the app not starting up > > > >> correctly or noticeable runtime errors. I did not check this: > perhaps > > > it is > > > >> happening more often than I think but with no side effects. This > could > > > >> happen if it sometimes outputs a typed method as instance.method() > > where > > > >> type resolution worked and elsewhere alongside as > instance['method']() > > > >> where it did not. The problem might not simply get noticed in this > > case, > > > >> but this is pure speculation, I have not checked for this. > > > >> > > > >> I did not try reducing heap allocation or anything to try to create > > > >> conditions for it to perhaps happen more often if it is memory/GC > > > related. > > > >> > > > >> I see notes like this in the code: > > > >> // If we get this far, then we did not find a cached entry > > > >> // It is possible for 2+ threads to get in here for the same name. > > > >> // This is intentional - the worst that happens is that we > duplicate > > > the > > > >> resolution work > > > >> // the benefit is that we avoid any sort of locking, which was > proving > > > >> expensive (time wise, > > > >> // and memory wise). > > > >> > > > >> When you see the code that was problematic output, you can see the > > same > > > >> name lookup inside a js method that is obviously correctly resolved > > > >> (anecdotally it seems to be more often 'correct' the first time) and > > > then > > > >> not wrong for subsequent output, in nearby code, so I assume it > might > > be > > > >> related to some unsynchronized state or failure to do that > 'duplicate' > > > >> resolution work, where the various parts were being processed in > > > parallel... > > > >> > > > >> Anyway, good luck, please let me know if you have anything you > think I > > > >> could do to help. > > > >> > > > >> > > > >> > > > >> On Tue, May 5, 2026 at 6:29 AM Harbs <[email protected]> wrote: > > > >> > > > >>> Sure. I’ll be in touch off list. > > > >>> > > > >>> > On May 4, 2026, at 9:18 PM, Josh Tynjala < > > [email protected]> > > > >>> wrote: > > > >>> > > > > >>> > Would you be willing to give me access to the project? If it > isn't > > > too > > > >>> > difficult to reproduce, I may be able to figure out what's going > on > > > >>> and how > > > >>> > to restore the missing typing data, similar to my other fix. My > > > >>> feeling is > > > >>> > that the original Adobe devs intended for occasional garbage > > > >>> collection to > > > >>> > occur to stay within memory limits, but that the data would be > > > >>> restorable, > > > >>> > if needed later. I think that they simply missed some places > where > > it > > > >>> might > > > >>> > need to be restored because it happens pretty rarely. Or maybe > our > > > >>> newer JS > > > >>> > emitter isn't properly accounting for that possibility. > > > >>> > > > > >>> > -- > > > >>> > Josh Tynjala > > > >>> > Bowler Hat LLC > > > >>> > https://bowlerhat.dev/ > > > >>> > > > > >>> > > > > >>> > On Mon, May 4, 2026 at 10:37 AM Harbs <[email protected]> > > wrote: > > > >>> > > > > >>> >>> You've tested that this issue still > > > >>> >>> reproduces using a compiler built from the latest source code? > > > >>> >> > > > >>> >> This was reproduced by a number of devs all working on the same > > > >>> project. > > > >>> >> And yes, it was with recent builds. > > > >>> >> > > > >>> >> I don’t think I personally have seen it (I have a lot of memory > on > > > my > > > >>> >> machine), but it seems to have gotten worse recently. I don’t > know > > > if > > > >>> >> something changed in the compiler or it’s due to the increased > > > >>> project size. > > > >>> >> > > > >>> >> This was with variables — not functions. > > > >>> >> > > > >>> >> Harbs > > > >>> >> > > > >>> >>> On May 4, 2026, at 6:54 PM, Josh Tynjala < > > > [email protected]> > > > >>> >> wrote: > > > >>> >>> > > > >>> >>> This issue may be the same one: > > > >>> >>> > > > >>> >>> https://github.com/apache/royale-compiler/issues/182 > > > >>> >>> > > > >>> >>> I also encountered and fixed an issue related weak references a > > > >>> little > > > >>> >> over > > > >>> >>> a year ago. Function bodies were getting garbage collected, > and I > > > >>> needed > > > >>> >> to > > > >>> >>> clear out some stale definitions that were causing missing > > classes > > > in > > > >>> >>> generated ASDoc output and some similar issues with the -watch > > > >>> compiler > > > >>> >>> option. > > > >>> >>> > > > >>> >>> > > > >>> >> > > > >>> > > > > > > https://github.com/apache/royale-compiler/commit/35eed62f13519c659e6346d26cca3f44afe3170f > > > >>> >>> > > > >>> >>> This fix does not appear to have made it into a release yet. > > You're > > > >>> not > > > >>> >>> using an older compiler build, right? You've tested that this > > issue > > > >>> still > > > >>> >>> reproduces using a compiler built from the latest source code? > > > >>> >>> > > > >>> >>> -- > > > >>> >>> Josh Tynjala > > > >>> >>> Bowler Hat LLC > > > >>> >>> https://bowlerhat.dev/ > > > >>> >>> > > > >>> >>> > > > >>> >>> On Sun, May 3, 2026 at 9:40 PM Greg Dove <[email protected]> > > > >>> wrote: > > > >>> >>> > > > >>> >>>> Compiler issues - (Josh, please?) > > > >>> >>>> > > > >>> >>>> We have a medium-sized project that has begun encountering > > > >>> >> occasional/rare > > > >>> >>>> (but at least daily during normal workloads) compilation > issues > > > that > > > >>> >> appear > > > >>> >>>> to be related to name/type resolution. There can be code > within > > a > > > >>> method > > > >>> >>>> output where the name resolves correctly to its type in one > part > > > of > > > >>> the > > > >>> >>>> method's js output and elsewhere within the same js method > > output > > > >>> as if > > > >>> >> it > > > >>> >>>> was Object/untyped. This is most obvious with XML or XMLList > > > >>> instances > > > >>> >>>> (because of .child('prop') vs ['prop] differences). I've also > > seen > > > >>> it > > > >>> >> get > > > >>> >>>> confused between local variables and instance properties in > some > > > >>> cases, > > > >>> >>>> which I believe is a manifestation of the same thing. In other > > > >>> words, > > > >>> >>>> different compilation runs with the exact same settings are > not > > > >>> >>>> completely deterministic, because sometimes they can provide > > > >>> different > > > >>> >>>> output. It is very difficult to repro, because it feels so > > random. > > > >>> But > > > >>> >> it > > > >>> >>>> has been something that appears to be more frequent as the > > > codebase > > > >>> >> grows > > > >>> >>>> (when all other settings remain the same). This led me to > > consider > > > >>> that > > > >>> >> it > > > >>> >>>> could be GC-related, and I recently removed the SoftReferences > > > >>> inside > > > >>> >>>> ASScopeCache, as a prime suspect. > > > >>> >>>> > > > >>> >>>> After doing this, I have not seen the problem since (so far - > > > after > > > >>> 1.5 > > > >>> >>>> days) > > > >>> >>>> > > > >>> >>>> The ASScopeCache instances themselves are weakly held (inside > > > >>> >>>> CompilerProject). So the internal maps inside each of these > > > >>> instances > > > >>> >> being > > > >>> >>>> weakly held as well seems to be the problem, the internal maps > > can > > > >>> >> perhaps > > > >>> >>>> get into a partially cleared state between threads. > > > >>> >>>> > > > >>> >>>> I did some memory profiling with and without this change for > > > >>> removing > > > >>> >> the > > > >>> >>>> SoftReferences inside ASScopeCache - but it was quite limited > > > (just > > > >>> >> testing > > > >>> >>>> with compiling the one project). The memory usage was not much > > > >>> >> different on > > > >>> >>>> a typical run (approx 1Mb difference for a compilation with > > around > > > >>> 1000 > > > >>> >> .as > > > >>> >>>> and .mxml files combined, alongside a bunch of local swcs). > > There > > > >>> was > > > >>> >>>> possibly a small speed up without the SoftReferences, but I > did > > > not > > > >>> test > > > >>> >>>> enough to be sure. > > > >>> >>>> But so far it seems there is not a big impact on memory with > > > >>> omitting > > > >>> >>>> these. If it introduces consistency I'm kinda keen to get it > in > > > >>> there - > > > >>> >> I > > > >>> >>>> know others have definitely seen this problem too. > > > >>> >>>> And for Josh in particular: I think your compiler experience > > > dwarfs > > > >>> the > > > >>> >>>> rest of us and wanted to get your feedback instead of just > > jumping > > > >>> in > > > >>> >> with > > > >>> >>>> this one. One option could also be to make this change as a > > > compiler > > > >>> >>>> option, with the new non-weak references being the default, > but > > > >>> with the > > > >>> >>>> ability to switch to the older behaviour via the option if > that > > > was > > > >>> >>>> considered important as well... look forward to hearing your > > > >>> thoughts. > > > >>> >>>> > > > >>> >> > > > >>> >> > > > >>> > > > >>> > > > > > >
