Hi everyone,
This week marks a major turning point. After fixing the critical UI
stability
issues in Week 6, we tackled the core challenge of populating our Object
Browser with live data. The journey led to a fundamental "aha!" moment about
how UNO's reflection system works, forcing a complete architectural rethink
of our data provider.
The result is a resounding success. We now have a stable, responsive Object
Browser that correctly populates the entire UNO API hierarchy on demand,
resolving the crashes and discovery issues that blocked us previously.
Gerrit Patch: https://gerrit.libreoffice.org/c/core/+/186822
Screen Shot Image:
https://bug-attachments.documentfoundation.org/attachment.cgi?id=201757
* == The Great Discovery Debate: A Flawed Assumption ==*
My initial approach to populating the UNO API tree was intuitive but, as it
turned out, fundamentally flawed. I assumed we could "discover" the API
hierarchically, making live API calls for each level of the tree as the
user expanded it.
**The Flawed "Live Discovery" Flow:**
+----------------------------------------------------------------------------+
| [User expands "UNO APIs"] -> Calls GetChildNodes("") |
| | |
| v |
|
+------------------------------------------------------------------------+ |
| | GetChildren("") | |
| | - SUCCESS: Returns hardcoded list: ["com", "org"] | |
|
+------------------------------------------------------------------------+ |
| | |
| [User expands "com"] -> Calls GetChildNodes("com") |
| | |
| v |
|
+------------------------------------------------------------------------+ |
| | GetChildren("com") | |
| | - Queries UNO API with "com". | |
| | - API returns an object, but *not* a list of child names. | |
| | - The check fails, an empty list is returned. | |
| | - RESULT: Tree expansion stops. -> BUG! | |
|
+------------------------------------------------------------------------+ |
+----------------------------------------------------------------------------+
This approach, using *XHierarchicalNameAccess::getByHierarchicalName*,
failed because the UNO reflection API is optimized for
Introspection (looking up a known, fully-qualified name), not for
hierarchical Discovery (browsing the contents of a partial path).
We were trying to ask the city for a list of all streets,
then ask each street for a list of all buildings—a process
that is inefficient and not directly supported.
* == The "Aha!" Moment: The Correct Architecture ==*
The breakthrough came from re-reading Stephan's crucial clarification on the
roles of the TypeManager and ServiceManager.
- ServiceManager: For component implementations.
- TypeManager: For all UNOIDL entity descriptions (types, services,
modules, etc.).
This clarified everything. The TypeManager holds the descriptions of
everything in the UNO API.
We cannot efficiently "discover" the tree live. Instead, we must first build
a map of the entire city and then use that map to guide the user.
This led to a complete redesign of the IdeDataProvider to use a hybrid
approach: a one-time cache for discovery, and live introspection for
details.
*The New, Correct Data Flow:*
+------------------------------------------------------------------+
| PHASE 1: ONE-TIME CACHE CREATION (in IdeDataProvider constructor)|
+------------------------------------------------------------------+
| |
| [1] Get the TypeDescriptionManager singleton. |
| ...getValueByName("...theTypeDescriptionManager") |
| |
| [2] Create an enumeration for ALL UNO entities. |
| ...createTypeDescriptionEnumeration(..., INFINITE) |
| |
| [3] Loop through the flat list of ~20,000 descriptions. |
| while (xEnum->hasMoreElements()) |
| |
| [4] For each description, parse its full name (e.g., |
| "com.sun.star.sheet.XSpreadsheet") and use addNode to |
| build a tree structure inside our m_hierarchyCache map. |
| |
| (This entire process takes ~0.9s and happens only once.) |
| |
+------------------------------------------------------------------+
|
v
+------------------------------------------------------------------+
| PHASE 2: LIVE UI INTERACTION |
+------------------------------------------------------------------+
| |
| [User expands "com.sun.star" in the TreeView] |
| | |
| v |
| [A] GetChildNodes("com.sun.star") is called. |
| - It performs an instant O(log N) lookup in our |
| m_hierarchyCache map. |
| - Returns the cached list of children (e.g., "sheet"). |
| |
| [User selects "XSpreadsheet" in the TreeView] |
| | |
| v |
| [B] GetMembers("...XSpreadsheet") is called. |
| - It performs a live introspection call. |
| - theCoreReflection->forName("...XSpreadsheet") |
| - This is fast and gets the rich member details. |
| |
+------------------------------------------------------------------+
This hybrid model gives us the best of both worlds: a fast, responsive UI
for browsing, and detailed, live information when the user needs it.
* == Q&A: The Architectural Deep Dive ==*
This new design was a deliberate choice based on our findings, and it's
worth clarifying the key principles.
*Q1: How does this reconcile with mentors saying "live fetching is not
slow"?*
A: The mentors were absolutely right, but we were misapplying the advice.
They were referring to Introspection (getting members of a single, known
type like XSpreadsheet), which our GetMembers function now does, and it
is very fast. Our initial error was trying to use a live API for
hierarchical Discovery, which is the part that is not directly supported
or performant. We now use the cache for discovery and live queries for
introspection.
***Q2: What is this "session cache" and how does it differ from the
long-term*
* IdeCodeCompletionCache vision?***
A: The cache we built this week is a session cache. It's a
std::map<OUString, SymbolInfoList> that lives only as long as the
IdeDataProvider instance. Its purpose is to solve the immediate functional
problem of making the UNO API browsable.
The long-term IdeCodeCompletionCache is a future goal for a **persistent
cache**. The logic we wrote to build our session cache could one day be run
during the LibreOffice build process to create a static file (e.g., SQLite)
that ships with the product. This would make startup even faster, as we'd be
loading a pre-computed tree instead of building it at runtime. Our current
work is the perfect prototype for that future system.
* Q3: What was the final `Reference.h` crash about?*
A: With the new data provider working, a final bug emerged: a crash when
selecting com.sun.star.beans.Ambiguous. Digging into the IDL file, I found
it's a generic template (struct Ambiguous<T>). Our code was too optimistic;
when introspecting this template, the reflection API would sometimes return
a NULL Reference<> for fields whose type depended on the unspecified
template parameter T. The fix was to make our code more defensive by adding
if (!xField.is()) and if (!xMethod.is()) checks in
ImplGetMembersOfUnoType, which has made the browser completely stable.
* == Current Status & Next Steps ==*
The Object Browser is now functional and stable. The left-hand tree
populates
correctly with identifying prefixes ([n], [I], [S], etc.), and the
right-hand pane shows the live members of any selected UNO entity.
With this solid foundation, we can now proceed with confidence:
- Immediate Next Step: Implement the right-hand members pane as a
TreeView with categories ("Properties", "Methods", "Events") to match
our design mockup.
- Then: Implement the bottom information pane to show the full
signature of the selected member.
This week was a huge leap forward, validating our architecture and
unblocking
the path to a feature-complete Object Browser.
Thanks for all the guidance and feedback that got us to this point.
I have also added a txt file in case the diagrams format went off direction.
Week 1 mail -
https://lists.freedesktop.org/archives/libreoffice/2025-May/093264.html
Week 2 and 3 mail -
https://lists.freedesktop.org/archives/libreoffice/2025-June/093362.html
Week 4 mail(Thread) -
https://lists.freedesktop.org/archives/libreoffice/2025-June/093392.html
Week 5 mail -
https://lists.freedesktop.org/archives/libreoffice/2025-June/093443.html
week 6 mail -
https://lists.freedesktop.org/archives/libreoffice/2025-July/093493.html
--
*Regards,*
*Devansh*
Hi everyone,
This week marks a major turning point. After fixing the critical UI stability
issues in Week 6, we tackled the core challenge of populating our Object
Browser with live data. The journey led to a fundamental "aha!" moment about
how UNO's reflection system works, forcing a complete architectural rethink
of our data provider.
The result is a resounding success. We now have a stable, responsive Object
Browser that correctly populates the entire UNO API hierarchy on demand,
resolving the crashes and discovery issues that blocked us previously.
Gerrit Patch: https://gerrit.libreoffice.org/c/core/+/186822
Screen Shot Image:
https://bug-attachments.documentfoundation.org/attachment.cgi?id=201757
== The Great Discovery Debate: A Flawed Assumption ==
My initial approach to populating the UNO API tree was intuitive but, as it
turned out, fundamentally flawed. I assumed we could "discover" the API
hierarchically, making live API calls for each level of the tree as the
user expanded it.
*The Flawed "Live Discovery" Flow:*
+----------------------------------------------------------------------------+
| [User expands "UNO APIs"] -> Calls GetChildNodes("") |
| | |
| v |
| +------------------------------------------------------------------------+ |
| | GetChildren("") | |
| | - SUCCESS: Returns hardcoded list: ["com", "org"] | |
| +------------------------------------------------------------------------+ |
| | |
| [User expands "com"] -> Calls GetChildNodes("com") |
| | |
| v |
| +------------------------------------------------------------------------+ |
| | GetChildren("com") | |
| | - Queries UNO API with "com". | |
| | - API returns an object, but *not* a list of child names. | |
| | - The check fails, an empty list is returned. | |
| | - RESULT: Tree expansion stops. -> BUG! | |
| +------------------------------------------------------------------------+ |
+----------------------------------------------------------------------------+
This approach, using XHierarchicalNameAccess::getByHierarchicalName,
failed because the UNO reflection API is optimized for
Introspection (looking up a known, fully-qualified name), not for
hierarchical Discovery (browsing the contents of a partial path).
We were trying to ask the city for a list of all streets,
then ask each street for a list of all buildings—a process
that is inefficient and not directly supported.
== The "Aha!" Moment: The Correct Architecture ==
The breakthrough came from re-reading Stephan's crucial clarification on the
roles of the TypeManager and ServiceManager.
- ServiceManager: For component implementations.
- TypeManager: For all UNOIDL entity descriptions (types, services,
modules, etc.).
This clarified everything. The TypeManager holds the descriptions of
everything in the UNO API.
We cannot efficiently "discover" the tree live. Instead, we must first build
a map of the entire city and then use that map to guide the user.
This led to a complete redesign of the IdeDataProvider to use a hybrid
approach: a one-time cache for discovery, and live introspection for details.
The New, Correct Data Flow:
+------------------------------------------------------------------+
| PHASE 1: ONE-TIME CACHE CREATION (in IdeDataProvider constructor)|
+------------------------------------------------------------------+
| |
| [1] Get the TypeDescriptionManager singleton. |
| ...getValueByName("...theTypeDescriptionManager") |
| |
| [2] Create an enumeration for ALL UNO entities. |
| ...createTypeDescriptionEnumeration(..., INFINITE) |
| |
| [3] Loop through the flat list of ~20,000 descriptions. |
| while (xEnum->hasMoreElements()) |
| |
| [4] For each description, parse its full name (e.g., |
| "com.sun.star.sheet.XSpreadsheet") and use addNode to |
| build a tree structure inside our m_hierarchyCache map. |
| |
| (This entire process takes ~0.9s and happens only once.) |
| |
+------------------------------------------------------------------+
|
v
+------------------------------------------------------------------+
| PHASE 2: LIVE UI INTERACTION |
+------------------------------------------------------------------+
| |
| [User expands "com.sun.star" in the TreeView] |
| | |
| v |
| [A] GetChildNodes("com.sun.star") is called. |
| - It performs an instant O(log N) lookup in our |
| m_hierarchyCache map. |
| - Returns the cached list of children (e.g., "sheet"). |
| |
| [User selects "XSpreadsheet" in the TreeView] |
| | |
| v |
| [B] GetMembers("...XSpreadsheet") is called. |
| - It performs a live introspection call. |
| - theCoreReflection->forName("...XSpreadsheet") |
| - This is fast and gets the rich member details. |
| |
+------------------------------------------------------------------+
This hybrid model gives us the best of both worlds: a fast, responsive UI
for browsing, and detailed, live information when the user needs it.
== Q&A: The Architectural Deep Dive ==
This new design was a deliberate choice based on our findings, and it's
worth clarifying the key principles.
Q1: How does this reconcile with mentors saying "live fetching is not slow"?
A: The mentors were absolutely right, but we were misapplying the advice.
They were referring to Introspection (getting members of a single, known
type like XSpreadsheet), which our GetMembers function now does, and it
is very fast. Our initial error was trying to use a live API for
hierarchical Discovery, which is the part that is not directly supported
or performant. We now use the cache for discovery and live queries for
introspection.
**Q2: What is this "session cache" and how does it differ from the long-term
IdeCodeCompletionCache vision?**
A: The cache we built this week is a session cache. It's a
std::map<OUString, SymbolInfoList> that lives only as long as the
IdeDataProvider instance. Its purpose is to solve the immediate functional
problem of making the UNO API browsable.
The long-term IdeCodeCompletionCache is a future goal for a **persistent
cache**. The logic we wrote to build our session cache could one day be run
during the LibreOffice build process to create a static file (e.g., SQLite)
that ships with the product. This would make startup even faster, as we'd be
loading a pre-computed tree instead of building it at runtime. Our current
work is the perfect prototype for that future system.
Q3: What was the final `Reference.h` crash about?
A: With the new data provider working, a final bug emerged: a crash when
selecting com.sun.star.beans.Ambiguous. Digging into the IDL file, I found
it's a generic template (struct Ambiguous<T>). Our code was too optimistic;
when introspecting this template, the reflection API would sometimes return
a NULL Reference<> for fields whose type depended on the unspecified
template parameter T. The fix was to make our code more defensive by adding
if (!xField.is()) and if (!xMethod.is()) checks in
ImplGetMembersOfUnoType, which has made the browser completely stable.
== Current Status & Next Steps ==
The Object Browser is now functional and stable. The left-hand tree populates
correctly with identifying prefixes ([n], [I], [S], etc.), and the
right-hand pane shows the live members of any selected UNO entity.
With this solid foundation, we can now proceed with confidence:
- Immediate Next Step: Implement the right-hand members pane as a
TreeView with categories ("Properties", "Methods", "Events") to match
our design mockup.
- Then: Implement the bottom information pane to show the full
signature of the selected member.
This week was a huge leap forward, validating our architecture and unblocking
the path to a feature-complete Object Browser.
Thanks for all the guidance and feedback that got us to this point.
I have also added a txt file in case the diagrams format went off direction.
Week 1 mail -
https://lists.freedesktop.org/archives/libreoffice/2025-May/093264.html
Week 2 and 3 mail -
https://lists.freedesktop.org/archives/libreoffice/2025-June/093362.html
Week 4 mail(Thread) -
https://lists.freedesktop.org/archives/libreoffice/2025-June/093392.html
Week 5 mail -
https://lists.freedesktop.org/archives/libreoffice/2025-June/093443.html
week 6 mail -
https://lists.freedesktop.org/archives/libreoffice/2025-July/093493.html
Regards,
Devansh
