Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin edited a discussion: [Discussion] The selection of Agentic/Taskflow frame ### Feature Description refer: https://blog.langchain.dev/how-to-think-about-agent-frameworks/ Some Context Here: Our goal at GraphRAG is to focus on providing better I/O capabilities related to LLM/RAG system. We need agentic but will not attempt to create a large and comprehensive (agent/rag) framework, so we tend to choose the highest performance/flexible workflow + agentic framework for integration(Any suggestions and feedback are welcome) Below is a simplified summary table assisted by LLM. Currently, the focus is on the first 4 (CrewAI / Agno / LLamaIndex / Pydanic-AI), with prioritization to be determined. They generally have their own built-in workflow-like designs and have few dependencies (relatively lightweight). Table: | Framework | Core Features | Documentation/Community Popularity | Workflow | Advantages | Disadvantages | GitHub ⭐️ | Lightweight | Notes | | --- | | | | | | | | | | [CrewAI](https://github.com/crewAIInc/crewAI) | Well-known multi-agent collaboration framework, supports sequential/hierarchical structures, dynamic workflow design, and dual-mode implementation of workflows | High | [Crew](https://docs.crewai.com/concepts/crews) + [Flow](https://docs.crewai.com/concepts/flows) design (the latter is event-driven) | Few dependencies, provides both Crew (role collaboration) + Flows (process control) usage modes, many [examples](https://github.com/crewAIInc/crewAI-examples) and documentation | 1. Requires understanding of Crew/Flow architecture and YAML configuration 2. Some tools may depend on Rust/C++ underlying libraries 3. Performance is unclear | 26.7k+ | Medium | Suitable for complex scenarios such as project management and strategic analysis, can be directly combined/compatible with LangChain/LLamaindex, disable [info collection](https://github.com/crewAIInc/crewAI/tree/main?tab=readme-ov-file#q -does-crewai-collect-any-data) when using | | [Agno](https://github.com/agno-agi/agno)| High-performance multimodal framework, supports text/image/audio/video, model-agnostic, extremely low memory footprint. Claims performance is X1000 times that of LangGraph | High | [workflow](https://docs.agno.com/workflows/introduction) no graph/no chain design, supports caching/persistence. Simply put, it does not provide syntactic sugar, the process is manually controlled, which is not very user-friendly | Extremely fast startup speed, native multimodal support, knowledge base integration | Low community maturity, relies on third-party model APIs | 19k+ | Medium-High | Claims memory usage is only 1/50 of LangGraph, suitable for high-concurrency scenarios | | [Llamaindex](https://github.com/run-llama/llama_index) | | High | Event-driven [workflow](https://docs.llamaindex.ai/en/stable/module_guides/workflow/) | Classic and established | | 39k+ | Medium-Low? | | | [Pydanic-AI](https://github.com/pydantic/pydantic-ai) | Produced by the Pydantic team, quality is relatively guaranteed, design should also be relatively good | Low | Classic [Graph](https://ai.pydantic.dev/graph/) implementation | New generation | Officially stated to be in Beta, not perfect enough, API changes at any time, use with caution in production | 6.5k+ | Medium-High? | | | [Lagent](https://github.com/InternLM/lagent)| Domestic lightweight multi-Agent framework, communication based on AgentMessage, defaults to providing synchronous/asynchronous interfaces | Low-Medium | Chained through the __call__ method | Lightweight + good domestic model support| Relatively basic functionality, limited multi-agent support, no
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin edited a discussion: [Discussion] The selection of Agentic/Taskflow frame ### Feature Description Update: https://blog.langchain.dev/how-to-think-about-agent-frameworks/ (_Note: not precise_) Some Context Here: Our goal at GraphRAG is to focus on providing better I/O capabilities related to LLM/RAG system. We need agentic but will not attempt to create a large and comprehensive (agent/rag) framework, so we tend to choose the highest performance/flexible workflow + agentic framework for integration(Any suggestions and feedback are welcome) Below is a simplified summary table assisted by LLM. Currently, the focus is on the first 4 (CrewAI / Agno / LLamaIndex / Pydanic-AI), with prioritization to be determined. They generally have their own built-in workflow-like designs and have few dependencies (relatively lightweight). Table: | Framework | Core Features | Documentation/Community Popularity | Workflow | Advantages | Disadvantages | GitHub ⭐️ | Lightweight | Notes | | --- | | | | | | | | | | [CrewAI](https://github.com/crewAIInc/crewAI) | Well-known multi-agent collaboration framework, supports sequential/hierarchical structures, dynamic workflow design, and dual-mode implementation of workflows | High | [Crew](https://docs.crewai.com/concepts/crews) + [Flow](https://docs.crewai.com/concepts/flows) design (the latter is event-driven) | Few dependencies, provides both Crew (role collaboration) + Flows (process control) usage modes, many [examples](https://github.com/crewAIInc/crewAI-examples) and documentation | 1. Requires understanding of Crew/Flow architecture and YAML configuration 2. Some tools may depend on Rust/C++ underlying libraries 3. Performance is unclear | 26.7k+ | Medium | Suitable for complex scenarios such as project management and strategic analysis, can be directly combined/compatible with LangChain/LLamaindex, disable [info collection](https://github.com/crewAIInc/crewAI/tree/main?tab=readme-ov-file#q -does-crewai-collect-any-data) when using | | [Agno](https://github.com/agno-agi/agno)| High-performance multimodal framework, supports text/image/audio/video, model-agnostic, extremely low memory footprint. Claims performance is X1000 times that of LangGraph | High | [workflow](https://docs.agno.com/workflows/introduction) no graph/no chain design, supports caching/persistence. Simply put, it does not provide syntactic sugar, the process is manually controlled, which is not very user-friendly | Extremely fast startup speed, native multimodal support, knowledge base integration | Low community maturity, relies on third-party model APIs | 19k+ | Medium-High | Claims memory usage is only 1/50 of LangGraph, suitable for high-concurrency scenarios | | [Llamaindex](https://github.com/run-llama/llama_index) | | High | Event-driven [workflow](https://docs.llamaindex.ai/en/stable/module_guides/workflow/) | Classic and established | | 39k+ | Medium-Low? | | | [Pydanic-AI](https://github.com/pydantic/pydantic-ai) | Produced by the Pydantic team, quality is relatively guaranteed, design should also be relatively good | Low | Classic [Graph](https://ai.pydantic.dev/graph/) implementation | New generation | Officially stated to be in Beta, not perfect enough, API changes at any time, use with caution in production | 6.5k+ | Medium-High? | | | [Lagent](https://github.com/InternLM/lagent)| Domestic lightweight multi-Agent framework, communication based on AgentMessage, defaults to providing synchronous/asynchronous interfaces | Low-Medium | Chained through the __call__ method | Lightweight + good domestic model support| Relatively basic functionality, limited
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin edited a discussion: [Discussion] The selection of Agentic/Taskflow frame ### Feature Description Update: https://blog.langchain.dev/how-to-think-about-agent-frameworks/ Some Context Here: Our goal at GraphRAG is to focus on providing better I/O capabilities related to LLM/RAG system. We need agentic but will not attempt to create a large and comprehensive (agent/rag) framework, so we tend to choose the highest performance/flexible workflow + agentic framework for integration(Any suggestions and feedback are welcome) Below is a simplified summary table assisted by LLM. Currently, the focus is on the first 4 (CrewAI / Agno / LLamaIndex / Pydanic-AI), with prioritization to be determined. They generally have their own built-in workflow-like designs and have few dependencies (relatively lightweight). Table: | Framework | Core Features | Documentation/Community Popularity | Workflow | Advantages | Disadvantages | GitHub ⭐️ | Lightweight | Notes | | --- | | | | | | | | | | [CrewAI](https://github.com/crewAIInc/crewAI) | Well-known multi-agent collaboration framework, supports sequential/hierarchical structures, dynamic workflow design, and dual-mode implementation of workflows | High | [Crew](https://docs.crewai.com/concepts/crews) + [Flow](https://docs.crewai.com/concepts/flows) design (the latter is event-driven) | Few dependencies, provides both Crew (role collaboration) + Flows (process control) usage modes, many [examples](https://github.com/crewAIInc/crewAI-examples) and documentation | 1. Requires understanding of Crew/Flow architecture and YAML configuration 2. Some tools may depend on Rust/C++ underlying libraries 3. Performance is unclear | 26.7k+ | Medium | Suitable for complex scenarios such as project management and strategic analysis, can be directly combined/compatible with LangChain/LLamaindex, disable [info collection](https://github.com/crewAIInc/crewAI/tree/main?tab=readme-ov-file#q -does-crewai-collect-any-data) when using | | [Agno](https://github.com/agno-agi/agno)| High-performance multimodal framework, supports text/image/audio/video, model-agnostic, extremely low memory footprint. Claims performance is X1000 times that of LangGraph | High | [workflow](https://docs.agno.com/workflows/introduction) no graph/no chain design, supports caching/persistence. Simply put, it does not provide syntactic sugar, the process is manually controlled, which is not very user-friendly | Extremely fast startup speed, native multimodal support, knowledge base integration | Low community maturity, relies on third-party model APIs | 19k+ | Medium-High | Claims memory usage is only 1/50 of LangGraph, suitable for high-concurrency scenarios | | [Llamaindex](https://github.com/run-llama/llama_index) | | High | Event-driven [workflow](https://docs.llamaindex.ai/en/stable/module_guides/workflow/) | Classic and established | | 39k+ | Medium-Low? | | | [Pydanic-AI](https://github.com/pydantic/pydantic-ai) | Produced by the Pydantic team, quality is relatively guaranteed, design should also be relatively good | Low | Classic [Graph](https://ai.pydantic.dev/graph/) implementation | New generation | Officially stated to be in Beta, not perfect enough, API changes at any time, use with caution in production | 6.5k+ | Medium-High? | | | [Lagent](https://github.com/InternLM/lagent)| Domestic lightweight multi-Agent framework, communication based on AgentMessage, defaults to providing synchronous/asynchronous interfaces | Low-Medium | Chained through the __call__ method | Lightweight + good domestic model support| Relatively basic functionality, limited multi-agent support, n
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Aryankb added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame @chiruu12 yes, that's a good idea. If we need to create a system, where user can create different agents, each agent will have it's own job , means it's own cypher-query to extract information from the graph. Some agents might do other work like information validator. Also , we can give user two options, either write own cypher-query, or generate query using LLM. In the second case, we might need fine tuning . GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666596 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > So summing it up here is the proposed architecture, kept simple: > > > Base Layer (Agno) > > Handle high-frequency L1 queries through optimized parallel execution > > Implement Gremlin-Cypher transpiler for hybrid query support > > Orchestration Layer (CrewAI) > > Manage complex workflows using dynamic intent classification > > Integrate with HugeGraph's priority queue system > > Validation Layer (Pydantic-AI) > > Enforce schema consistency across all graph operations > > Provide developer-friendly type hints > > Retrieval Enhancement (LlamaIndex) > > Implement recursive retrieval with tiered caching > > Integrate with HugeGraph's OLAP engine > > My rationale and research summarized: Agno delivers performance gains for > core operations while maintaining lower memory usage CrewAI's workflow engine > reduces development time for complex agent interactions compared to manual > implementations Hybrid model achieves a much higher fault recovery through > layered fallback mechanisms > > This proposed architecture is based off of what I saw on the apache's jira, > where the required architecture was provided for the upcoming months of > development. > > I also emailed you additional insights for the architecture, please do check > ( > [@imbajin](https://github.com/imbajin?rgh-link-date=2025-03-01T06%3A01%3A39.000Z) > ) @Kryst4lDem0ni4s Thanks for ur graph, and I have fixed the mermaid UI: https://github.com/user-attachments/assets/d487762d-3aa1-457d-869a-46561d7ef38e"; /> And some comment about the graph: https://github.com/user-attachments/assets/ae555a2c-32b2-4a2d-b322-282565877f99"; /> As I mentioned above, it is entirely feasible to mix the strengths of multiple frameworks, but we are unlikely to directly introduce the entire framework (such as `pip install xxx`). Instead, we should choose the source code that can be extracted separately and integrate it directly into our tool (of course, we need to follow ASF's reference specifications). If that feature is bundled with multiple frameworks and difficult to separate separately, we may need to evaluate its importance. We are likely to first introduce a relatively balanced framework and integrate the good aspects of other designs for transformation. This is currently my thought GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1217 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Kryst4lDem0ni4s added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame Okay, I'll get right to it and let you know! Also, I'll come back afterwards to help document this discussion thoroughly once I have improved the overall understanding of the concepts. Thanks for the reference GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1222 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Aryankb added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame @Kryst4lDem0ni4s i liked your idea of caching and prometheus monitoring But using different frameworks for different tasks comes up with too many dependency issues. Once i was working with crew ai, and langchain-google-genai library, it took me two days to fix the dependency conflicts by manually tring which version combination works. So it might give hybrid features but its very inconvenient and could lead to Dependency hell. GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1208 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Kryst4lDem0ni4s added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame By hybrid approaches, I mean something like: CrewAI's NLP pipeline enhanced with HugeGraph's domain-specific embeddings Agno's parallel processing for high-volume L1 requests GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1204 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > integrating only the critical components from these frameworks? @chiruu12 It would be best if we could do this, not be bound by unnecessary design of the framework, it will also be easier to switch and iterate over time (in fact the `Agentic GraphRAG` I understand is still similar to a `single-Agent` framework, and the logic should be kept as clear and concise as possible) GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666591 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > I see, so then I think it's feasible to side with CrewAI and build further > with a manual implementation of the other libraries. I also think that once > the features are fully implemented with crewAI, we can isolate some items for > optimizing the performance using Agno's principles. CrewAI here, being the > preference since it has far more to offer as compared to Agno and its primary > strength being its efficiency. > > Especially since we have to handle L1-LN queries sooner or later, we'll > iteratively improve it using the others (Agno, LlamaIndex, etc.) Any > suggestions of where we can start with the implementation of the demo, > because it would be best to test it out first before finalizing the idea? > [@imbajin](https://github.com/imbajin?rgh-link-date=2025-03-03T02%3A50%3A38.000Z) @Kryst4lDem0ni4s As for how to try it out, I think you can follow the general process below (for reference only): 1. First, fully understand the current reading process(Next is the writing process), including the `text2gql` pattern (simple)+extracting keywords ->matching vid -->BFS(multistep) to look up subgraph pattern. Of course, it would be even better if we improved this documentation to clarify everyone's previous context clearer 2. Understanding our current `Pipeline` design, it is relatively simple, just a multi-step separation similar to a factory design pattern(fixed) 3. Try the flow part of the `crewai` and write a simple demo to be familiar with its usage (there is no need to introduce the entire crew design at this time) 4. Then, combining with our GraphRAG process, we will attempt to introduce a minimal agent designed crewai example and simulate it as simply as possible 5. Then checkout a new branch to see how our current version integrates with minimal changes in architecture, and execute it step by step to avoid making too many modifications at once that are difficult to debug 6. . GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1221 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame @imbajin Sir btw have you tried smolagents?? GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666599 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame @Aryankb I think most of the people won't be proficient enough to write their own queries I have worked quite a bit with graph rag in my intern and at first even I had a bit trouble in writing those. So, I would suggest that we can get a description for the knowledge that the user will be providing us if they don't, we will by default use a LLM to get what the knowledge or text is about and then make an agent write the query for us and use that query ? @imbajin sir what is your opinion on this? GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666597 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > I looked further into what [@chiruu12](https://github.com/chiruu12) suggests, > about not using off-the-shelf agentic components that can prevent developers > from understanding critical behaviors, so that over-time the behavior of the > service doesnt go out of control. Indeed it would be possible to write a > custom HG_agentic library that borrows only the necessary pieces, so > hugegraph can maintain control over the logic and integration details. > > How about combining all of our suggestions that integrates LlamaIndex, > Pydantic-AI, CrewFlow, and Agno into a dual-mode, modular GraphRAG system > while avoiding the dependency hell as warned by > [@Aryankb](https://github.com/Aryankb) > > As a hybrid GraphRAG system that supports two modes we can include: • A > beginner-friendly “agentic retriever” that is pre-fine-tuned with robust LLM > prompting for straightforward use cases, and • A customizable mode for > advanced developers who need to tailor retrieval, orchestration, and > validation mechanisms. > > Key design principles so that everyone can get a good night's sleep: • > Modularity & Microservices: standalone services with clearly defined APIs. • > Dual-Mode Operation: ease of use and deep customization. • Transparent > Integration: extracting core functionalities and integrating them in-house. • > Extensive Logging & Monitoring: via Prometheus. • Containerization: isolate > dependencies. > > Architectural Layers & Components: A. Base Layer – Agno for L1 Queries Handle > high-frequency, low-latency queries (e.g., simple entity lookups) with > optimized parallel execution. Beyond this point we must also seek the correct > strategy to handle LN queries as well. > > Key Features: • Fast execution with low memory footprint. • Built-in > Gremlin-Cypher transpiler for hybrid query support. • Integration with a > hybrid caching layer that combines Agno shared memory and RocksDB. • Wrap > Agno’s core query engine in a microservice that exposes an HTTP endpoint. • > Queries can be configured to pass through a lightweight pre-processing step > to select between cache and live query execution (L1) specific. > > This component when abstracted into our own agentic library will be the base > of all performance optimizations. > > B. Orchestration Layer – CrewAI for Complex Workflows This would help us > manage multi-hop, dynamic queries and agent workflows that require intent > classification and asynchronous execution and allows customization. > > Key Features: • Dynamic intent classification powered by domain-specific > embeddings (integrated with HugeGraph). • Event-driven workflow, where > subtasks are dynamically generated from a user’s plain-English prompt. • > Built-in support for sequencing (sequential/parallel) and conditional > delegation of agent tasks. • Adapt core functionalities from CrewAI > (CrewFlow) to create a custom orchestration module. • Define a clear API > contract for submitting workflows, retrieving status, and handling > error/fallback logic. > > C. Validation Layer – Pydantic All general schema consistency and data > integrity across all operations. THIS distinction is necessary to understand > that it's sole purpose here should be for schema purposes only, not beyond it > so far. > > Key Features: • Middleware to validate incoming queries and agent responses. > • Dev-friendly type hints and error reporting. • Mechanisms to ensure that > changes in one layer do not break API contracts. • Wrap core endpoints of > other layers with Pydantic models that perform input/output validation. • > Integrate validation middleware as a separate microservice or as decorators > within the existing service codebase. > > Note: This is the general usage of Pydantic, not it's agentic tools. > Otherwise it is too unpredictable and unsuitable for production. > > D. Retrieval Enhancement Layer – LlamaIndex Finally, in order to provide > recursive, multi-hop retrieval functionality enhanced by tiered caching, > ensuring that complex graph queries are answered effectively, LlamaIndex is > compatible with CrewAI alreayd, so we'll look further into how it's > compatibility has been provided. Key Features: • Recursive retrieval > strategies that work well with hierarchical graph caching. • Integration with > HugeGraph’s OLAP engine for analytical queries. • Modular “runnables” > inspired by LangChain that allow flexible composition of retrieval steps. • > Expose LlamaIndex’s retrieval engine via an API that accepts complex, > multi-hop query parameters. • Use a caching strategy that combines in-memory > (for fast lookups) and persistent (RocksDB) storage to accelerate repeated > queries. > > Summary for the plan with general key points and implementation steps: > > * RESTful API endpoints for query submission, workflow orchestrat
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Kryst4lDem0ni4s added a comment to the discussion: [Discussion] The
selection of Agentic/Taskflow frame
@imbajin , I agree with @Aryankb's understanding but as for the workflow
priority, here are my two cents:
```mermaid
graph TD
A[User Query] --> B(Agno L1 Processor)
B -->|Simple Lookup| C[HugeGraph Cache]
B -->|Complex Query| D{CrewAI Orchestrator}
D -->|Multi-Hop| E[LlamaIndex Retriever]
D -->|Computation| F[HugeGraph-Computer]
F -->G
E --> G[Result Aggregator]
G --> H[Pydantic Validator]
H --> I[Output Formatter]
```
Here, we'll be considering the top four candidate frameworks (CrewAI, Agno,
LlamaIndex, Pydantic-AI) against HugeGraph's requirements for implementing an
agentic GraphRAG system.
Instead of taking one out of the bunch, I'll reiterate my suggestion for the
hybrid approach.
Deploy Agno for L1 queries
Implement CrewAI's dynamic classifier with HugeGraph embeddings
Develop hybrid caching layer (RocksDB + Agno shared memory)
Build Pydantic validation middleware
Introduce LlamaIndex recursive retrieval
CrewAI's Performance Profile (I asked chatgpt for an analysis)
```
Throughput: 8,500 QPS (L2 queries)
Memory: 1.8GB per orchestration node
Latency: 45ms P99 for complex workflows
Key Advantages
Native integration with HugeGraph's RocksDB-based embeddings
Prebuilt Prometheus metrics exporter for OLAP/OLTP monitoring
```
So summing it up here is the proposed architecture, kept simple:
> Base Layer (Agno)
> Handle high-frequency L1 queries through optimized parallel execution
> Implement Gremlin-Cypher transpiler for hybrid query support
> Orchestration Layer (CrewAI)
> Manage complex workflows using dynamic intent classification
> Integrate with HugeGraph's priority queue system
> Validation Layer (Pydantic-AI)
> Enforce schema consistency across all graph operations
> Provide developer-friendly type hints
> Retrieval Enhancement (LlamaIndex)
> Implement recursive retrieval with tiered caching
> Integrate with HugeGraph's OLAP engine
My rationale and research summarized:
Agno delivers performance gains for core operations while maintaining lower
memory usage
CrewAI's workflow engine reduces development time for complex agent
interactions compared to manual implementations
Hybrid model achieves a much higher fault recovery through layered fallback
mechanisms
This proposed architecture is based off of what I saw on the apache's jira,
where the required architecture was provided for the upcoming months of
development.
I also emailed you additional insights for the architecture, please do check (
@imbajin )
GitHub link:
https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1207
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > So for finding the answer of user's question, I used LANGGRAPH. It was really > lightweight , Fast and simple to implement it. gpt uh!! GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666594 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > Also > [@imbajin](https://github.com/imbajin?rgh-link-date=2025-02-28T07%3A26%3A17.000Z) > sir just wanted to know , what is the actual requirement. > > * Is it direct prompt -->information > (where user just write a prompt, subtasks will be divided and agents gets > created based on the knowledge graph structure) > * or user will have capability to create and orchestrate his own agents ( to > extract specific information from graph) > > And also where we are currently ? Good question in fact, we need to provide both of these abilities at the same time, but with a **focus on the second point**. We can understand that the first one is mainly aimed at **novice users**(novice here means: people who are not completely unfamiliar with **property graphs**, and they don't expect a way where simply throwing a `text/PDF` can make everything goes fine (similar to simple/one-click but casual extraction of RDF/triplets), and we need to simply support this case, but it is not the key (similarly, including visualization/UI/compatibility with different vector-DBs, etc., are not our focus) Our core focus is on devs with basic vector-db(vector-rag/naive/basic-rag) or Agent systems. Assuming they already have Vector-RAG, how can we better integrate GraphRAG to provide more operability and ease of orchestration? This is also why we provide separate `HTTP-API` layer encapsulation, which facilitates developers to call our core query functions directly. For example, suppose the user is already using `AutoGen` to orchestrate their Agent system, and now assuming that the data in the graph has been extracted (skipping the extraction step), how can they better and faster integrate it into their original system, while maintaining high performance and simplicity as much as possible They(Devs) can directly modify our pipeline/workflow code, instead of requiring us to provide a fixed "local/global" mode like Microsoft GraphRAG that is not easy to adjust. Our agentization/selection is about how to better and faster achieve this goal. I'm not sure if the explanation is clear, but if there are any questions, please continue to reply:) BTW, we are currently in a transition from 1 to 2 GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1202 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame @imbajin yes sir and as most of these frameworks are open source and them being already lightweight it is easy to go through the whole code though it will take some time, we can use their code at the places where we need I do have a understanding of Agno framework once we decide which framework we will be using I will go through the whole code for the same for the time being I will try to understand the code for hugegraph-ai I went through the docs and will be trying it out. I hope I will be useful in making this project a level above from what you expect (btw I replied lightning quick because I was checking the docs!). My understanding is a little different for the Agentic GraphRag sir. I believe for increasing the accuracy we will have to take some extra steps in the agent (like a manager or validator/ fallback kind of agent) so that it is able to make decisions more accurately, for that we will have to settle for the optimal trade-off for the accuracy and time. GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666592 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Kryst4lDem0ni4s added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame Modifying and creating your own implementations of existing frameworks is quite impractical on such a scale, especially when your reasoning is for a small upgrade of performance optimisation. It's better to use existing libraries and the features they offer instead of creating something from scratch, otherwise it's a separate project altogether. As for dependency hell, it is definitely true that we'll have to spend some time initially to figure out compatibility between all the projects, but once the initial investment is cleared, we'll be able to achieve better productivity in the long run as the project scales. I firmly believe in micro-service-based architecture and loose coupling between different services, that's the same rationale here. It's easy to utilise the best of all their facilities (CrewAI, Agno, LlamaIndex and Pydantic) if we define the modularity from the get go. So I'd say that from here on out, it's not much of an issue about "which is the best?" Because we've already figured that part out. Now we need to focus on "which is best for which component?". More specifically, "What goes where?". Let's focus on defining a scalable architecture and not shy away from experimentation when we have such great resources available. And building something from scratch for minimal performance upgrades is not a wise investment unless we can come up with a way to achieve what Agno achieved (thousands of times better in performance). GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1211 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > have thought of a simpler solution for the same and also I will be talking > to agno's team to get some insights too > [@imbajin](https://github.com/imbajin?rgh-link-date=2025-03-02T02%3A22%3A35.000Z) > sir please let me know if you will be up for a little discussion for the > same. @chiruu12 Fine, I am usually able to respond to messages in my personal time, but I may not be able to reply in time~ GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1218 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > [@Aryankb](https://github.com/Aryankb?rgh-link-date=2025-02-28T07%3A12%3A17.000Z) > I think most of the people won't be proficient enough to write their own > queries I have worked quite a bit with graph rag in my intern and at first > even I had a bit trouble in writing those. So, I would suggest that we can > get a description for the knowledge that the user will be providing us if > they don't, we will by default use a LLM to get what the knowledge or text is > about and then make an agent write the query for us and use that query ? > [@imbajin](https://github.com/imbajin?rgh-link-date=2025-02-28T07%3A12%3A17.000Z) > sir what is your opinion on this? @chiruu12 @Aryankb First, regarding the `text2gql` part, it is an independent matter, and I understand that it is not strongly related to the selection of agentic frame or workflow impl. Here is a brief description of the actual situation. Our implementation and approach earlier was to use both model fine-tuning and **user templates** simultaneously. (see it ↓ By default, we use the GQL query template to optimize the effect of text2gql.) https://github.com/user-attachments/assets/fc278898-4cbf-46b4-8d4d-90dcf0e7df6d"; /> General encoder model fine-tuning for `7-14B` can be a significant task, especially when it comes to how to generate GQL corpus (HG uses Gremlin queries by default and is compatible with most of the Cypher syntax), refer [wiki](https://github.com/apache/incubator-hugegraph-ai/wiki/HugeGraph-LLM-Roadmap#4-graph-query-core-1) to get more context GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1201 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame [Pydantic-AI](https://github.com/pydantic/pydantic-ai) could have been an excellent option for us. However, since it's still in beta, adopting it would require updating our codebase, which would increase our workload. Therefore, I suggest going with Agno. That said, Agno's manually controlled workflow has its own pros and cons, so we will need to make some compromises with that approach as well. GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666589 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Aryankb added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame Also @imbajin sir just wanted to know , what is the actual requirement. - Is it direct prompt -->information (where user just write a prompt, subtasks will be divided and agents gets created based on the knowledge graph structure) - or user will have capability to create and orchestrate his own agents ( to extract specific information from graph) And also where we are currently ? GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666598 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The
selection of Agentic/Taskflow frame
> [@imbajin](https://github.com/imbajin?rgh-link-date=2025-02-28T20%3A47%3A30.000Z)
> As per my understanding of 2, we need a way to integrate hugegraph agents
> with any other existing framework. And we will provide an http api layer
> abstraction or python SDK for hugegraph agents. For example. Someone working
> on some projects involving AI agents. They thought that they needs to use
> GRAPH RAG for some use case. Now for that, they will be able to import HG
> agentic library say HG_agentic. Now using this library, they can create
> agents using plane english (what the agent needs to do, given some input and
> required output format.) Each agent will have the information about graph
> from which information needs to be extracted. And it will have some inputs -
> (output from previous agents, like node names , relationship names ,
> similarity search results, etc), the agent will convert txt2gql according to
> the given prompt and graph structure knowledge. This way (Devs) can create
> agents and orchestrat
e them using our library HG_orchestrator by just passing all created agents in
a data structure.
>
> HG_orchestrator will handle the order of agent execution. User can specify if
> he want sequential in case next agent needs outputs from any previous agent ,
> parallel otherwise.
>
> If we need this, we need a framework having really good `WORKFLOW FEATURES`.
> Because then we can be really flexible. (We can manage the flow of data
> between agents, as specified by user, also we can add if-else validation
> condition, deligation steps like deligate to previous agent with some
> modified input.)
>
> Eg :- `HG_orchestrate([agent1, agent2, agent3], fllow="sequential",
> data_flow: {agent1 : [agent2, agent3], agent2 : [agent3]}, deligation:
> [{from:agent3, to:agent1 , condition :"some_condition", modification :
> "required_modification"}]`
>
> One naive approach can be to do everything using simple python while loop,
> here we don't need to manage different dependency conflicts. If (devs) use
> AutoGen, and if we create HG_agentic SDK using crewai. Then there might be
> dependency conflicts.
>
> If not, then we can follow the below priority order.
>
> Priority suggestion for workflow :-
>
> * 1. Llamaindex (have workflow features like checkpoint, trigger, retries,
> streaming, context across runs, with really good example as per our needs)
> * 2. pydantic AI
> * 3. CrewFlow
> * 4. Agno (don't have workflow features)
@Aryankb Yes, I basically agree with your hypothesis and analysis. The best way
for us to proceed is to provide an SDK for developers to better call, although
that is not something that needs to be done immediately.
And also as the
https://github.com/apache/incubator-hugegraph-ai/issues/183#issuecomment-2692091406
mentioned We may try a more balanced option first. If we feel that the agent
framework is heavy, we can directly reference/use their flow part instead of
directly referencing the entire framework (after looking at the workflows of
`Crewai` and `Llamaindex`, we can independently propose them)
GitHub link:
https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1215
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Kryst4lDem0ni4s added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > As I mentioned above, it is entirely feasible to mix the strengths of > multiple frameworks, but we are unlikely to directly introduce the entire > framework (such as `pip install xxx`). Instead, we should choose the source > code that can be extracted separately and integrate it directly into our tool > (of course, we need to follow ASF's reference specifications). If that > feature is bundled with multiple frameworks and difficult to separate > separately, we may need to evaluate its importance. > > We are likely to first introduce a relatively balanced framework and > integrate the good aspects of other designs for transformation. This is > currently my thought I see, so then I think it's feasible to side with CrewAI and build further with a manual implementation of the other libraries. I also think that once the features are fully implemented with crewAI, we can isolate some items for optimizing the performance using Agno's principles. CrewAI here, being the preference since it has far more to offer as compared to Agno and its primary strength being its efficiency. Especially since we have to handle L1-LN queries sooner or later, we'll iteratively improve it using the others (Agno, LlamaIndex, etc.) Any suggestions of where we can start with the implementation of the demo, because it would be best to test it out first before finalizing the idea? @imbajin GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1220 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Kryst4lDem0ni4s added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame I looked further into what @chiruu12 suggests, about not using off-the-shelf agentic components that can prevent developers from understanding critical behaviors, so that over-time the behavior of the service doesnt go out of control. Indeed it would be possible to write a custom HG_agentic library that borrows only the necessary pieces, so hugegraph can maintain control over the logic and integration details. How about combining all of our suggestions that integrates LlamaIndex, Pydantic-AI, CrewFlow, and Agno into a dual-mode, modular GraphRAG system while avoiding the dependency hell as warned by @Aryankb As a hybrid GraphRAG system that supports two modes we can include: • A beginner-friendly “agentic retriever” that is pre-fine-tuned with robust LLM prompting for straightforward use cases, and • A customizable mode for advanced developers who need to tailor retrieval, orchestration, and validation mechanisms. Key design principles so that everyone can get a good night's sleep: • Modularity & Microservices: standalone services with clearly defined APIs. • Dual-Mode Operation: ease of use and deep customization. • Transparent Integration: extracting core functionalities and integrating them in-house. • Extensive Logging & Monitoring: via Prometheus. • Containerization: isolate dependencies. Architectural Layers & Components: A. Base Layer – Agno for L1 Queries Handle high-frequency, low-latency queries (e.g., simple entity lookups) with optimized parallel execution. Beyond this point we must also seek the correct strategy to handle LN queries as well. Key Features: • Fast execution with low memory footprint. • Built-in Gremlin-Cypher transpiler for hybrid query support. • Integration with a hybrid caching layer that combines Agno shared memory and RocksDB. • Wrap Agno’s core query engine in a microservice that exposes an HTTP endpoint. • Queries can be configured to pass through a lightweight pre-processing step to select between cache and live query execution (L1) specific. This component when abstracted into our own agentic library will be the base of all performance optimizations. B. Orchestration Layer – CrewAI for Complex Workflows This would help us manage multi-hop, dynamic queries and agent workflows that require intent classification and asynchronous execution and allows customization. Key Features: • Dynamic intent classification powered by domain-specific embeddings (integrated with HugeGraph). • Event-driven workflow, where subtasks are dynamically generated from a user’s plain-English prompt. • Built-in support for sequencing (sequential/parallel) and conditional delegation of agent tasks. • Adapt core functionalities from CrewAI (CrewFlow) to create a custom orchestration module. • Define a clear API contract for submitting workflows, retrieving status, and handling error/fallback logic. C. Validation Layer – Pydantic All general schema consistency and data integrity across all operations. THIS distinction is necessary to understand that it's sole purpose here should be for schema purposes only, not beyond it so far. Key Features: • Middleware to validate incoming queries and agent responses. • Dev-friendly type hints and error reporting. • Mechanisms to ensure that changes in one layer do not break API contracts. • Wrap core endpoints of other layers with Pydantic models that perform input/output validation. • Integrate validation middleware as a separate microservice or as decorators within the existing service codebase. Note: This is the general usage of Pydantic, not it's agentic tools. Otherwise it is too unpredictable and unsuitable for production. D. Retrieval Enhancement Layer – LlamaIndex Finally, in order to provide recursive, multi-hop retrieval functionality enhanced by tiered caching, ensuring that complex graph queries are answered effectively, LlamaIndex is compatible with CrewAI alreayd, so we'll look further into how it's compatibility has been provided. Key Features: • Recursive retrieval strategies that work well with hierarchical graph caching. • Integration with HugeGraph’s OLAP engine for analytical queries. • Modular “runnables” inspired by LangChain that allow flexible composition of retrieval steps. • Expose LlamaIndex’s retrieval engine via an API that accepts complex, multi-hop query parameters. • Use a caching strategy that combines in-memory (for fast lookups) and persistent (RocksDB) storage to accelerate repeated queries. Summary for the plan with general key points and implementation steps: - RESTful API endpoints for query submission, workflow orchestration, validation, and retrieval. - A Python SDK (e.g., HG_agentic and HG_orchestrator) that abstracts away the internal microservices and provides simple functions for (examples): Creating agents via plain-English commands. Configuring custom
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Kryst4lDem0ni4s added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame Although I have already started the next steps, I will be working on creating a deeper implementation and the logic extraction from the crewai's library based on this implementation of their available code. 1. https://github.com/Kryst4lDem0ni4s/incubator-hugegraph-ai/blob/agenticprototype/hugegraph-llm/src/hugegraph_llm/operators/graph_rag_task.py 2. https://github.com/Kryst4lDem0ni4s/incubator-hugegraph-ai/blob/agenticprototype/hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/graph_rag_query.py - In the code file links shared above, I focused on first trying to define how the agentic and workflow setup should be logically (very basic). Also note that this implemntation is focused on trying to augment the current code, instead of replacing anything. - In the next step, I will use this basic implementation, and utilize the open-source code available from crewai, to be able to create our own functionality handlers (to replace import requirements completely). - Also as you can see, I created a very simple file (or class) based implementation of the requirements (agent and flow core concepts in crewai), so the next focus would be to create a global orchestrator that is able to handle the complete workflow altogether, creating a true agentic system, not limited to a single workflow. - Afterwards, we can dive deeper into the successive requirements like figuring out the deeper logical requirements, implementing compatible logic from other libraries, etc. Whenever possible, please go through the implementation and please provide feedback so that I can detect mistakes made in the pipeline @imbajin GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1223 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > [@imbajin](https://github.com/imbajin?rgh-link-date=2025-02-28T22%3A01%3A46.000Z) > sir instead of developing a dedicated HG-agentic library, I propose that we > make a retriever service similar to what Pinecone provides. Our approach > would encompass two distinct modes: a beginner-friendly “agentic retriever” > that comes pre-fine-tuned with a robust LLM using few-shot or one-shot > prompting (can also integrating advanced prompting techniques and dynamic > re-ranking for accurate graph query generation) a fully customizable > retriever mode aimed at developers. In the customizable mode, users can > modify key code placeholders—such as LLM selection, prompt configuration, and > integration of additional tools to replace or enhance their existing RAG > systems, including those orchestrated by frameworks like AutoGen. Moreover, > we can architect this retriever module using a modular “runnables” concept, > like we have in LangChain’s design, which ensures framework agnosticism and > ease of inte gration into various vector database ecosystems (e.g., Pinecone, FAISS, Qdrant). By also providing an HTTP API layer for direct access to our core query functions, this solution will not only offer high performance and simplicity but also enables integration with existing agent systems, making it an ideal bridge between novice users and expert developers. I think this solution will be covering both the points and help the people from both the levels Also enabling beginners as they will be able to use our service without much hassle. the end note is I am inclined to making the agent from scratch rather than taking it as it is from an existing framework and as almost all of the frameworks are opensource we can take the core functionality from them and add those to our own structure. @chiruu12 I think it can be done this way. We can eventually implement our own management of agent combinations, but we still need a good workflow library (if we feel that the performance is not good enough, we can consider using Rust to rewrite the underlying layer - similar to [Pydanic-core](https://github.com/pydantic/pydantic-core)). We just need to ensure consistency in the user interface layer GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1216 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user imbajin added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > Hey > [@imbajin](https://github.com/imbajin?rgh-link-date=2025-02-28T06%3A39%3A43.000Z) > I was working with agentic graph RAG recently. The project aimed at Document > QnA from a story book. It required Agentic graph-RAG approach as in a story, > different characters can have different relationships with each other. Also > different incidents are linked to each other. So, here agentic graph-RAG > worked well if someone asks deep questions from story. So i made graph where > centre node was book --> first level nodes were chunks (each chapter of that > book) --> second level nodes were atomic_facts from that chunk--> final level > nodes were characters. Same characters from different atomic_facts were > connected (so as to know that character was present in which parts of the > story). Also different characters were connected to each other having some > relationships. > > So for finding the answer of user's question, I used LANGGRAPH. It was really > lightweight , Fast and simple to implement it. > > There were agents - > > * character finder (to find all characters, user's question contains) > * atomic fact finder ( worked at finding the relevant atomic facts for each > character, according to user query. Using similarity search) > * information validator (checked if information is enough) > * atomic fact extractor (if information is not enough, then extract the > nearby atomic fact, or nearby events in that story) > * final composer (compose final answer using extracted information and user > query. > > It was a fixed agentic system, worked at graphs of one kind only > > I have a question, regarding what is the requirement ? Do we need to create a > Interface, where user can create his own agentic system for his Graph , just > using drag-drop or prompts (without code) If yes, then using LANGGRAPH or > completely doing it manually would be the best approach, because there will > be transparency , and user would know what is happening. Using CrewAI would > not be a good idea , as there we don't know what is happening under the hood. > > Here is the project :- > [Aryankb/DOC_QNA](https://github.com/Aryankb/DOC_QNA?rgh-link-date=2025-02-28T06%3A39%3A43.000Z) @Aryankb Thanks for your feedback. At present, whether we use `crewai`, `llamaindex`, or `pydanic-ai`, we should mainly use its **workflow component** & some basic agent encapsulation, and it is likely that we will **not use role concepts** such as `Crew` for design, If pydanic-ai is relatively stable, it actually looks like one of the most suitable (it also has a Graph design similar to LangGraph) The main concern about LangGraph comes from feedback that its performance is poor and its resource consumption is high(Like `Agno`'s benchmark). In addition, it is often used in combination with `LangChain`. We do not want to passively introduce the LangChain family bucket(And in fact we already added an `HugeGraphQAChain` interface in it..) GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1200 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame I believe for our use case we will have to train to model to also get us the text-to-cypher? As we wont know what kind of input the user is going to give us so for that we will have to fine tune a model for the same as the agent might need to write the cypher query on it's own? > Hey [@imbajin](https://github.com/imbajin) I was working with agentic graph > RAG recently. The project aimed at Document QnA from a story book. It > required Agentic graph-RAG approach as in a story, different characters can > have different relationships with each other. Also different incidents are > linked to each other. So, here agentic graph-RAG worked well if someone asks > deep questions from story. So i made graph where centre node was book --> > first level nodes were chunks (each chapter of that book) --> second level > nodes were atomic_facts from that chunk--> final level nodes were characters. > Same characters from different atomic_facts were connected (so as to know > that character was present in which parts of the story). Also different > characters were connected to each other having some relationships. > > So for finding the answer of user's question, I used LANGGRAPH. It was really > lightweight , Fast and simple to implement it. > > There were agents - > > * character finder (to find all characters, user's question contains) > * atomic fact finder ( worked at finding the relevant atomic facts for each > character, according to user query. Using similarity search) > * information validator (checked if information is enough) > * atomic fact extractor (if information is not enough, then extract the > nearby atomic fact, or nearby events in that story) > * final composer (compose final answer using extracted information and user > query. > > It was a fixed agentic system, worked at graphs of one kind only > > I have a question, regarding what is the requirement ? Do we need to create a > Interface, where user can create his own agentic system for his Graph , just > using drag-drop or prompts (without code) If yes, then using LANGGRAPH or > completely doing it manually would be the best approach, because there will > be transparency , and user would know what is happening. Using CrewAI would > not be a good idea , as there we don't know what is happening under the hood. > > Here is the project :- https://github.com/Aryankb/DOC_QNA GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666595 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame @imbajin sir I went through all the repositories for the agentic frameworks you have mentioned above sorry took quite a while in doing that. I believe we should be looking at crewai or agno only. I believe there are significant opportunities to reduce their weight by removing unnecessary components for our use and we could potentially create a more streamlined solution by extracting just the essential components we need. Would you agree with taking a more minimalist approach by identifying and integrating only the critical components from these frameworks? Also i am working on a small demo of how an ai agent can be made for the same can i get a few api call examples for it? so that I can use those examples in my demo? GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666590 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user vichayturen added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame Currently, I've only reviewed the workflow aspects of `llamaindex` and `crewai`. Both tools exhibit similar performance and syntax for building workflows. In terms of usage, they follow an event-driven model and support asynchronous task construction. Based on their coding characteristics, my conclusions are: - `llamaindex` offers higher flexibility but results in more disorganized code. - `crewai` has a cleaner design but more limited functionality. The **Graph Rag** workflow in `incubator-hugegraph-llm` combines the **text2gql** and **subgraph query** approaches. The workflow-building capabilities of the two aforementioned tools can cover the current logic, so I lean toward `crewai` for its more streamlined workflow implementation. Additionally, in my view: The immediate benefit of introducing **Agent** and **Task** concepts is to better manage prompt templates, so adopting these should primarily consider performance and dependency management. Next steps: 1. Integrate these concepts to build a fully fixed Graph Rag workflow with API access. 2. Strategic priorities: Should the focus be on improving flexibility or enhancing the existing Graph Rag workflow? - If prioritizing flexibility: Should the goal be to a) simplify creating new workflows, b) ease adjustments to workflow composition/sequence, or c) enable easy overrides of individual flow implementations? These questions require further consideration. GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1210 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > [@Kryst4lDem0ni4s](https://github.com/Kryst4lDem0ni4s) i liked your idea of > caching and prometheus monitoring > > But using different frameworks for different tasks comes up with too many > dependency issues. Once i was working with crew ai, and > langchain-google-genai library, it took me two days to fix the dependency > conflicts by manually tring which version combination works. So it might give > hybrid features but its very inconvenient and could lead to Dependency hell. @Aryankb first of all good point and second this is one of the reasons why I suggested taking the features and writing our own as we can also try to work with the latest dependencies for our framework and it will be not lightweight anymore if will end up using so many frameworks (one of the points that sir made in the starting) Also not understand the code and just using them without getting to the bottom of it might cause problems which is bound to happen if we are going to use so many frameworks I would suggest using pyndatic (not the agentic framework) and a combination of agno and crewai in which we will be building our own agent (not exactly our own because we will be just modifying their code and using for our framework). It will ensure that we know what is happening and when and even if there is a component that is not fitting well, we will know why it is not fitting well, and we can make the required changes instead of being dependent on the framework we will be choosing. GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1209 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > > They(Devs) can directly modify our pipeline/workflow code, instead of > > requiring us to provide a fixed "local/global" mode like Microsoft GraphRAG > > that is not easy to adjust. > > [@imbajin](https://github.com/imbajin) As per my understanding of 2, we need > a way to integrate hugegraph agents with any other existing framework. And we > will provide an http api layer abstraction or python SDK for hugegraph > agents. For example. Someone working on some projects involving AI agents. > They thought that they needs to use GRAPH RAG for some use case. Now for > that, they will be able to import HG agentic library say HG_agentic. Now > using this library, they can create agents using plane english (what the > agent needs to do, given some input and required output format.) Each agent > will have the information about graph from which information needs to be > extracted. And it will have some inputs - (output from previous agents, like > node names , relationship names , similarity search results, etc), the agent > will convert txt2gql according to the given prompt and graph structure > knowledge. This way (Devs) can create agents and orchestrate them using our > library HG_orchestrator by just passing all created agents in a data structure. @imbajin sir instead of developing a dedicated HG-agentic library, I propose that we make a retriever service similar to what Pinecone provides. Our approach would encompass two distinct modes: a beginner-friendly “agentic retriever” that comes pre-fine-tuned with a robust LLM using few-shot or one-shot prompting (can also integrating advanced prompting techniques and dynamic re-ranking for accurate graph query generation) a fully customizable retriever mode aimed at developers. In the customizable mode, users can modify key code placeholders—such as LLM selection, prompt configuration, and integration of additional tools to replace or enhance their existing RAG systems, including those orchestrated by frameworks like AutoGen. Moreover, we can architect this retriever module using a modular “runnables” concept, like we have in LangChain’s design, which ensures framework agnosticism and ease of integration into various vector database ecosystems (e.g., Pinecone, FAISS, Qdrant). By also providing an HTTP API layer for direct access to our core query functions, this solution will not only offer high performance and simplicity but also enables integration with existing agent systems, making it an ideal bridge between novice users and expert developers. I think this solution will be covering both the points and help the people from both the levels Also enabling beginners as they will be able to use our service without much hassle. the end note is I am inclined to making the agent from scratch rather than taking it as it is from an existing framework and as almost all of the frameworks are opensource we can take the core functionality from them and add those to our own structure. GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1206 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Aryankb added a comment to the discussion: [Discussion] The
selection of Agentic/Taskflow frame
> For example, suppose the user is already using `AutoGen` to orchestrate their
> Agent system, and now assuming that the data in the graph has been extracted
> (skipping the extraction step), how can they better and faster integrate it
> into their original system, while maintaining high performance and simplicity
> as much as possible
> They(Devs) can directly modify our pipeline/workflow code, instead of
> requiring us to provide a fixed "local/global" mode like Microsoft GraphRAG
> that is not easy to adjust.
@imbajin As per my understanding of 2, we need a way to integrate hugegraph
agents with any other existing framework. And we will provide an http api layer
abstraction or python SDK for hugegraph agents.
For example. Someone working on some projects involving AI agents. They thought
that they needs to use GRAPH RAG for some use case. Now for that, they will be
able to import HG agentic library say HG_agentic. Now using this library, they
can create agents using plane english (what the agent needs to do, given some
input and required output format.) Each agent will have the information about
graph from which information needs to be extracted. And it will have some
inputs - (output from previous agents, like node names , relationship names ,
similarity search results, etc), the agent will convert txt2gql according to
the given prompt and graph structure knowledge. This way (Devs) can create
agents and orchestrate them using our library HG_orchestrator by just passing
all created agents in a data structure.
HG_orchestrator will handle the order of agent execution. User can specify if
he want sequential in case next agent needs outputs from any previous agent ,
parallel otherwise.
If we need this, we need a framework having really good `WORKFLOW FEATURES`.
Because then we can be really flexible. (We can manage the flow of data between
agents, as specified by user, also we can add if-else validation condition,
deligation steps like deligate to previous agent with some modified input.)
Eg :-
`HG_orchestrate([agent1, agent2, agent3], fllow="sequential", data_flow:
{agent1 : [agent2, agent3], agent2 : [agent3]}, deligation: [{from:agent3,
to:agent1 , condition :"some_condition", modification :
"required_modification"}]`
One naive approach can be to do everything using simple python while loop, here
we don't need to manage different dependency conflicts. If (devs) use AutoGen,
and if we create HG_agentic SDK using crewai. Then there might be dependency
conflicts.
If not, then we can follow the below priority order.
Priority suggestion for workflow :-
- 1. Llamaindex (have workflow features like checkpoint, trigger, retries,
streaming, context across runs, with really good example as per our needs)
- 2. pydantic AI
- 3. CrewFlow
- 4. Agno (don't have workflow features)
GitHub link:
https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1205
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Kryst4lDem0ni4s added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame @imbajin I would have to suggest either CrewAI or Agno. CrewAI may have a higher memory overhead compared to baseline systems. CrewAI's NLP pipeline with HugeGraph's domain-specific embeddings can improve dynamic intent recognition. Agno's parallel processing capabilities are beneficial for handling high-volume L1 requests efficiently. I'm researching this more in detail, especially looking at the project's upcoming goals. Simply put: CrewAI: Pros: Native support for mixed synchronous/asynchronous execution Prebuilt integrations with Prometheus monitoring Cons: 15-20% higher memory overhead compared to baseline Agno claims performance improvements up to 1000 times that of traditional frameworks but the integration may require custom adapters for HugeGraph's Gremlin/Cypher hybrid interface. This seems like a solid investment into the performance. Adding to the above, LlamaIndex provides a graph-aware retrieval system with a recursive mechanism for hierarchical caching AND It integrates seamlessly with tools like CrewAI to enhance search-based queries and agentic pipelines. Frameworks with active communities and regular updates, like CrewAI and LlamaIndex are practical for use, but if we're leaning more towards a DIY approach, with a focus on performance, Agno should be prioritized. Also to be noted, LlamaIndex MAY lack native support for HugeGraph's distributed computer module, need to look further into it. CrewAI seems like the perfect option, memory efficiency aside. Memory efficiency considered, Agno must be looked into. How about hybrid approaches? GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1203 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > > [@imbajin](https://github.com/imbajin?rgh-link-date=2025-02-28T22%3A01%3A46.000Z) > > sir instead of developing a dedicated HG-agentic library, I propose that > > we make a retriever service similar to what Pinecone provides. Our approach > > would encompass two distinct modes: a beginner-friendly “agentic retriever” > > that comes pre-fine-tuned with a robust LLM using few-shot or one-shot > > prompting (can also integrating advanced prompting techniques and dynamic > > re-ranking for accurate graph query generation) a fully customizable > > retriever mode aimed at developers. In the customizable mode, users can > > modify key code placeholders—such as LLM selection, prompt configuration, > > and integration of additional tools to replace or enhance their existing > > RAG systems, including those orchestrated by frameworks like AutoGen. > > Moreover, we can architect this retriever module using a modular > > “runnables” concept, like we have in LangChain’s design, which ensures > > framework agnosticism and ease of in tegration into various vector database ecosystems (e.g., Pinecone, FAISS, Qdrant). By also providing an HTTP API layer for direct access to our core query functions, this solution will not only offer high performance and simplicity but also enables integration with existing agent systems, making it an ideal bridge between novice users and expert developers. I think this solution will be covering both the points and help the people from both the levels Also enabling beginners as they will be able to use our service without much hassle. the end note is I am inclined to making the agent from scratch rather than taking it as it is from an existing framework and as almost all of the frameworks are opensource we can take the core functionality from them and add those to our own structure. > > [@chiruu12](https://github.com/chiruu12) I think it can be done this way. We > can eventually implement our own management of agent combinations, but we > still need a good workflow library (if we feel that the performance is not > good enough, we can consider using Rust to rewrite the underlying layer - > similar to [Pydanic-core](https://github.com/pydantic/pydantic-core)). We > just need to ensure consistency in the user interface layer Sure, sir I will make a demo for the same as soon as my exams are over sir. GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1219 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user chiruu12 added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame > I looked further into what [@chiruu12](https://github.com/chiruu12) suggests, > about not using off-the-shelf agentic components that can prevent developers > from understanding critical behaviors, so that over-time the behavior of the > service doesnt go out of control. Indeed it would be possible to write a > custom HG_agentic library that borrows only the necessary pieces, so > hugegraph can maintain control over the logic and integration details. @Kryst4lDem0ni4s glad to know that you did get my point there actually I am currently looking at the code for the specs I have in mind which is a little different as I would prefer a more simple but easier to understand implementation (atleast for the prototype I want to make currently). As even though these frameworks are considered lightweight understanding each and every component, will take a bit of time also I have my semester exams so will be busy for the next two weeks in that. Hope we can all learn from each other and will be able to make a good solution for the same. GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-1213 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
Re: [D] [Discussion] The selection of Agentic/Taskflow frame [incubator-hugegraph-ai]
GitHub user Aryankb added a comment to the discussion: [Discussion] The selection of Agentic/Taskflow frame Hey @imbajin I was working with agentic graph RAG recently. The project aimed at Document QnA from a story book. It required Agentic graph-RAG approach as in a story, different characters can have different relationships with each other. Also different incidents are linked to each other. So, here agentic graph-RAG worked well if someone asks deep questions from story. So i made graph where centre node was book --> first level nodes were chunks (each chapter of that book) --> second level nodes were atomic_facts from that chunk--> final level nodes were characters. Same characters from different atomic_facts were connected (so as to know that character was present in which parts of the story). Also different characters were connected to each other having some relationships. So for finding the answer of user's question, I used LANGGRAPH. It was really lightweight , Fast and simple to implement it. There were agents - - character finder (to find all characters, user's question contains) - atomic fact finder ( worked at finding the relevant atomic facts for each character, according to user query. Using similarity search) - information validator (checked if information is enough) - atomic fact extractor (if information is not enough, then extract the nearby atomic fact, or nearby events in that story) - final composer (compose final answer using extracted information and user query. It was a fixed agentic system, worked at graphs of one kind only I have a question, regarding what is the requirement ? Do we need to create a Interface, where user can create his own agentic system for his Graph , just using drag-drop or prompts (without code) If yes, then using LANGGRAPH or completely doing it manually would be the best approach, because there will be transparency , and user would know what is happening. Using CrewAI would not be a good idea , as there we don't know what is happening under the hood. Here is the project :- https://github.com/Aryankb/DOC_QNA GitHub link: https://github.com/apache/incubator-hugegraph-ai/discussions/203#discussioncomment-12666593 This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
