PragmaTwice commented on code in PR #223: URL: https://github.com/apache/kvrocks-website/pull/223#discussion_r1623465271
########## blog/2024-06-02-kqir-query-engine/index.md: ########## @@ -0,0 +1,222 @@ +--- +slug: kqir-query-engine +title: "KQIR: a query engine for Apache Kvrocks that supports both SQL and RediSearch queries" +authors: [twice] +--- + +## Intro + +TL;DR: + + + +Pretty cool, right? Let's dive in! + +<!--truncate--> + +### Apache Kvrocks + +[Apache Kvrocks](https://kvrocks.apache.org/) is a [Redis](https://redis.io/)-compatible database built on [RocksDB](https://rocksdb.org/). + +It supports [the RESP protocol](https://redis.io/docs/latest/develop/reference/protocol-spec/) (version 2 and 3) and [a wide range of Redis commands](/docs/supported-commands), encompassing core data structures like Strings, Sets, Hashes, Sorted Sets, Stream, GEO, as well as Lua Scripts, Transactions, [Functions](https://redis.io/docs/latest/develop/interact/programmability/functions-intro/) and even [BloomFilter](https://redis.io/docs/latest/develop/data-types/probabilistic/bloom-filter/), [JSON](https://redis.io/docs/latest/develop/data-types/json/) from the Redis Stack. + +Unlike Redis which stores data in memory, Kvrocks persists data on disk for improved storage capabilities without being constrained by machine memory limit. + +### The capability to query + +In recent decades, NoSQL databases have gained prominence over traditional databases for their superior performance, scalability, and versatility across various industries. + +However, many users are reluctant to completely forego the essential features of SQL databases just for performance reasons. +These include ACID transactions, expressive query capabilities inherent in SQL, as well as optimization and abstraction possibilities offered by structured data and relational algebra. +Consequently, a new category of databases known as NewSQL has emerged gradually. + +Kvrocks is a NoSQL database. +While not classified as NewSQL, Kvrocks aims to strike a balance between NoSQL and NewSQL paradigms: +It endeavors to uphold the high performance associated with NoSQL while bolstering transactional guarantees alongside supporting more expressive query methods. + +### RediSearch? + +[RediSearch](https://github.com/RediSearch/RediSearch) is a Redis module that enhances Redis with query, secondary indexing, and full-text search functionalities. +While [its Redis commands](https://redis.io/docs/latest/operate/oss_and_stack/stack-with-enterprise/search/commands/) begin with `FT.` (i.e. full text), it goes beyond just full-text search. + +In fact, it is Redis moving closer to SQL databases: +RediSearch enables users to create structured schemas on existing Redis JSON or HASH data for index building. +Its schema supports [various field types](https://redis.io/docs/latest/develop/interact/search-and-query/basic-constructs/field-and-type-options/) such as numeric, tag, geo, text, and vector - the latter two are utilized for full-text and vector searches. +Instead of SQL support, RediSearch provides [a unique query syntax](https://redis.io/docs/latest/develop/interact/search-and-query/advanced-concepts/query_syntax/) known as the RediSearch query language. + +RediSearch finds applications in various fields. +One recent application involves utilizing its vector search feature to develop retrieval-augmented generation (RAG). For instance, [LangChain](https://www.langchain.com/) utilizes Redis as one of its vector database. +If Kvrocks can be compatible with RediSearch, it could benefit from these ecosystem from RediSearch. + +### SQL? + +RediSearch uses a unique syntax for queries, but there are some issues to consider: + +Firstly, RediSearch's schema (known as an index created with `FT.CREATE`) can be likened to a table in an SQL database. Its query syntax also aligns semantically with SQL queries. +Given this similarity, supporting SQL should not pose significant challenges; why not incorporate it? + +Secondly, SQL enjoys broader usage and is familiar to more individuals. It is simpler to grasp and learn at the syntax level. While developers may need time to understand RediSearch query syntax when using it, adapting to a new SQL database typically requires less effort. Furthermore, SQL offers robust support for various query features, enhanced expressive capabilities (like JOINs, subqueries, aggregation), and greater scalability. Review Comment: Hmm seems just a random word, removed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
