Sounds good Josh, I will do that. I haven't heard feedback yet from day one of others testing the patched compiler. I will wait one more day, (still with my fingers crossed) before I even assume that it works.
On Thu, May 14, 2026 at 4:48 AM Josh Tynjala <[email protected]> wrote: > Thanks, Greg. I guess a branch is good, in case I'd like to suggest any > tweaks. > > -- > Josh Tynjala > Bowler Hat LLC > https://bowlerhat.dev/ > > > On Tue, May 12, 2026 at 10:10 PM Greg Dove <[email protected]> wrote: > > > Josh, I *think* it might be a combination of the two. I'm asking others > who > > were seeing it more often than I did to test a possible fix (I will share > > an updated compiler build with them), because repro of the actual issue > is > > still quite challenging. > > *if* it works (maybe will need 1-2 days to be sure), I'll push it either > > directly to dev or via branch/PR (lmk what you prefer) and I'd certainly > > appreciate your review of that if possible. I did use a bit of AI support > > for the sleuthing and the testing, but you have spent much more time in > the > > compiler codebase than I have. > > For now, though, fingers crossed.... > > > > > > > > On Wed, May 13, 2026 at 4:00 AM Josh Tynjala <[email protected]> > > wrote: > > > > > Thanks for the update, Greg. Threading could certainly be a cause if we > > > arre missing some kind of synchronization. I know that we have > > > workspace.startBuilding() and workspace.startIdleState() as ways of > > > ensuring threads are under control. We may be missing one of those > calls > > > somewhere before emitting JS. > > > > > > As for GC, I recall that reducing JVM memory wasn't necessarily enough > > for > > > me to reproduce the other GC related bug I mentioned, strange as that > > > seems. I remember also adding System.gc() calls in various places > > (though I > > > don't remember exactly where), and I think that's what finally allowed > me > > > to reproduce the issue semi-reliably. > > > > > > -- > > > Josh Tynjala > > > Bowler Hat LLC > > > https://bowlerhat.dev/ > > > > > > > > > On Mon, May 11, 2026 at 10:15 PM Greg Dove <[email protected]> > wrote: > > > > > > > Hey Josh, > > > > > > > > I am actively looking into this again. I am less convinced that it is > > GC > > > > related (I reduced memory allocation to low levels) and perhaps it is > > > more > > > > to do with threads/race-conditions. But it's very difficult to be > > sure, I > > > > spent today adding logging and trying to repro, but did not repro the > > bug > > > > all day. I will keep on this tomorrow trying to find the right > > conditions > > > > to force it to occur. If I can figure out what those are, I will > share > > > them > > > > with you. > > > > > > > > -Greg > > > > > > > > > > > > On Tue, May 5, 2026 at 9:06 AM Greg Dove <[email protected]> > wrote: > > > > > > > > > Wake up brain (self talk): > > > > > "and then not wrong for subsequent output" <- should be of course > > "and > > > > > then wrong for subsequent output". > > > > > > > > > > On Tue, May 5, 2026 at 9:05 AM Greg Dove <[email protected]> > > wrote: > > > > > > > > > >> Thanks for looking into this, Josh. > > > > >> > > > > >> "If it isn't too difficult to reproduce" > > > > >> Quick comments, just in case it helps: > > > > >> > > > > >> It was not something I could repro for debugging purposes in the > > > > >> compiler. It was still 'rare' in practice - max 2-3 times per day > > > that I > > > > >> observed, sometimes only once a day - and not manifesting in the > > same > > > > code > > > > >> - although perhaps that is simply because code can change a lot > > > between > > > > >> compiler runs - and "awareness" was based on the app not starting > up > > > > >> correctly or noticeable runtime errors. I did not check this: > > perhaps > > > > it is > > > > >> happening more often than I think but with no side effects. This > > could > > > > >> happen if it sometimes outputs a typed method as instance.method() > > > where > > > > >> type resolution worked and elsewhere alongside as > > instance['method']() > > > > >> where it did not. The problem might not simply get noticed in this > > > case, > > > > >> but this is pure speculation, I have not checked for this. > > > > >> > > > > >> I did not try reducing heap allocation or anything to try to > create > > > > >> conditions for it to perhaps happen more often if it is memory/GC > > > > related. > > > > >> > > > > >> I see notes like this in the code: > > > > >> // If we get this far, then we did not find a cached entry > > > > >> // It is possible for 2+ threads to get in here for the same name. > > > > >> // This is intentional - the worst that happens is that we > > duplicate > > > > the > > > > >> resolution work > > > > >> // the benefit is that we avoid any sort of locking, which was > > proving > > > > >> expensive (time wise, > > > > >> // and memory wise). > > > > >> > > > > >> When you see the code that was problematic output, you can see the > > > same > > > > >> name lookup inside a js method that is obviously correctly > resolved > > > > >> (anecdotally it seems to be more often 'correct' the first time) > and > > > > then > > > > >> not wrong for subsequent output, in nearby code, so I assume it > > might > > > be > > > > >> related to some unsynchronized state or failure to do that > > 'duplicate' > > > > >> resolution work, where the various parts were being processed in > > > > parallel... > > > > >> > > > > >> Anyway, good luck, please let me know if you have anything you > > think I > > > > >> could do to help. > > > > >> > > > > >> > > > > >> > > > > >> On Tue, May 5, 2026 at 6:29 AM Harbs <[email protected]> > wrote: > > > > >> > > > > >>> Sure. I’ll be in touch off list. > > > > >>> > > > > >>> > On May 4, 2026, at 9:18 PM, Josh Tynjala < > > > [email protected]> > > > > >>> wrote: > > > > >>> > > > > > >>> > Would you be willing to give me access to the project? If it > > isn't > > > > too > > > > >>> > difficult to reproduce, I may be able to figure out what's > going > > on > > > > >>> and how > > > > >>> > to restore the missing typing data, similar to my other fix. My > > > > >>> feeling is > > > > >>> > that the original Adobe devs intended for occasional garbage > > > > >>> collection to > > > > >>> > occur to stay within memory limits, but that the data would be > > > > >>> restorable, > > > > >>> > if needed later. I think that they simply missed some places > > where > > > it > > > > >>> might > > > > >>> > need to be restored because it happens pretty rarely. Or maybe > > our > > > > >>> newer JS > > > > >>> > emitter isn't properly accounting for that possibility. > > > > >>> > > > > > >>> > -- > > > > >>> > Josh Tynjala > > > > >>> > Bowler Hat LLC > > > > >>> > https://bowlerhat.dev/ > > > > >>> > > > > > >>> > > > > > >>> > On Mon, May 4, 2026 at 10:37 AM Harbs <[email protected]> > > > wrote: > > > > >>> > > > > > >>> >>> You've tested that this issue still > > > > >>> >>> reproduces using a compiler built from the latest source > code? > > > > >>> >> > > > > >>> >> This was reproduced by a number of devs all working on the > same > > > > >>> project. > > > > >>> >> And yes, it was with recent builds. > > > > >>> >> > > > > >>> >> I don’t think I personally have seen it (I have a lot of > memory > > on > > > > my > > > > >>> >> machine), but it seems to have gotten worse recently. I don’t > > know > > > > if > > > > >>> >> something changed in the compiler or it’s due to the increased > > > > >>> project size. > > > > >>> >> > > > > >>> >> This was with variables — not functions. > > > > >>> >> > > > > >>> >> Harbs > > > > >>> >> > > > > >>> >>> On May 4, 2026, at 6:54 PM, Josh Tynjala < > > > > [email protected]> > > > > >>> >> wrote: > > > > >>> >>> > > > > >>> >>> This issue may be the same one: > > > > >>> >>> > > > > >>> >>> https://github.com/apache/royale-compiler/issues/182 > > > > >>> >>> > > > > >>> >>> I also encountered and fixed an issue related weak > references a > > > > >>> little > > > > >>> >> over > > > > >>> >>> a year ago. Function bodies were getting garbage collected, > > and I > > > > >>> needed > > > > >>> >> to > > > > >>> >>> clear out some stale definitions that were causing missing > > > classes > > > > in > > > > >>> >>> generated ASDoc output and some similar issues with the > -watch > > > > >>> compiler > > > > >>> >>> option. > > > > >>> >>> > > > > >>> >>> > > > > >>> >> > > > > >>> > > > > > > > > > > https://github.com/apache/royale-compiler/commit/35eed62f13519c659e6346d26cca3f44afe3170f > > > > >>> >>> > > > > >>> >>> This fix does not appear to have made it into a release yet. > > > You're > > > > >>> not > > > > >>> >>> using an older compiler build, right? You've tested that this > > > issue > > > > >>> still > > > > >>> >>> reproduces using a compiler built from the latest source > code? > > > > >>> >>> > > > > >>> >>> -- > > > > >>> >>> Josh Tynjala > > > > >>> >>> Bowler Hat LLC > > > > >>> >>> https://bowlerhat.dev/ > > > > >>> >>> > > > > >>> >>> > > > > >>> >>> On Sun, May 3, 2026 at 9:40 PM Greg Dove < > [email protected]> > > > > >>> wrote: > > > > >>> >>> > > > > >>> >>>> Compiler issues - (Josh, please?) > > > > >>> >>>> > > > > >>> >>>> We have a medium-sized project that has begun encountering > > > > >>> >> occasional/rare > > > > >>> >>>> (but at least daily during normal workloads) compilation > > issues > > > > that > > > > >>> >> appear > > > > >>> >>>> to be related to name/type resolution. There can be code > > within > > > a > > > > >>> method > > > > >>> >>>> output where the name resolves correctly to its type in one > > part > > > > of > > > > >>> the > > > > >>> >>>> method's js output and elsewhere within the same js method > > > output > > > > >>> as if > > > > >>> >> it > > > > >>> >>>> was Object/untyped. This is most obvious with XML or XMLList > > > > >>> instances > > > > >>> >>>> (because of .child('prop') vs ['prop] differences). I've > also > > > seen > > > > >>> it > > > > >>> >> get > > > > >>> >>>> confused between local variables and instance properties in > > some > > > > >>> cases, > > > > >>> >>>> which I believe is a manifestation of the same thing. In > other > > > > >>> words, > > > > >>> >>>> different compilation runs with the exact same settings are > > not > > > > >>> >>>> completely deterministic, because sometimes they can provide > > > > >>> different > > > > >>> >>>> output. It is very difficult to repro, because it feels so > > > random. > > > > >>> But > > > > >>> >> it > > > > >>> >>>> has been something that appears to be more frequent as the > > > > codebase > > > > >>> >> grows > > > > >>> >>>> (when all other settings remain the same). This led me to > > > consider > > > > >>> that > > > > >>> >> it > > > > >>> >>>> could be GC-related, and I recently removed the > SoftReferences > > > > >>> inside > > > > >>> >>>> ASScopeCache, as a prime suspect. > > > > >>> >>>> > > > > >>> >>>> After doing this, I have not seen the problem since (so far > - > > > > after > > > > >>> 1.5 > > > > >>> >>>> days) > > > > >>> >>>> > > > > >>> >>>> The ASScopeCache instances themselves are weakly held > (inside > > > > >>> >>>> CompilerProject). So the internal maps inside each of these > > > > >>> instances > > > > >>> >> being > > > > >>> >>>> weakly held as well seems to be the problem, the internal > maps > > > can > > > > >>> >> perhaps > > > > >>> >>>> get into a partially cleared state between threads. > > > > >>> >>>> > > > > >>> >>>> I did some memory profiling with and without this change for > > > > >>> removing > > > > >>> >> the > > > > >>> >>>> SoftReferences inside ASScopeCache - but it was quite > limited > > > > (just > > > > >>> >> testing > > > > >>> >>>> with compiling the one project). The memory usage was not > much > > > > >>> >> different on > > > > >>> >>>> a typical run (approx 1Mb difference for a compilation with > > > around > > > > >>> 1000 > > > > >>> >> .as > > > > >>> >>>> and .mxml files combined, alongside a bunch of local swcs). > > > There > > > > >>> was > > > > >>> >>>> possibly a small speed up without the SoftReferences, but I > > did > > > > not > > > > >>> test > > > > >>> >>>> enough to be sure. > > > > >>> >>>> But so far it seems there is not a big impact on memory with > > > > >>> omitting > > > > >>> >>>> these. If it introduces consistency I'm kinda keen to get it > > in > > > > >>> there - > > > > >>> >> I > > > > >>> >>>> know others have definitely seen this problem too. > > > > >>> >>>> And for Josh in particular: I think your compiler experience > > > > dwarfs > > > > >>> the > > > > >>> >>>> rest of us and wanted to get your feedback instead of just > > > jumping > > > > >>> in > > > > >>> >> with > > > > >>> >>>> this one. One option could also be to make this change as a > > > > compiler > > > > >>> >>>> option, with the new non-weak references being the default, > > but > > > > >>> with the > > > > >>> >>>> ability to switch to the older behaviour via the option if > > that > > > > was > > > > >>> >>>> considered important as well... look forward to hearing your > > > > >>> thoughts. > > > > >>> >>>> > > > > >>> >> > > > > >>> >> > > > > >>> > > > > >>> > > > > > > > > > >
