This is an automated email from the ASF dual-hosted git repository. chenliang613 pushed a commit to branch chenliang613-patch-2 in repository https://gitbox.apache.org/repos/asf/carbondata.git
commit b1c126befcadc5baefb6b840da7c1e89202859d0 Author: Liang Chen <[email protected]> AuthorDate: Mon Oct 6 18:23:56 2025 +0800 Enhance README with AI-native data storage details Expanded on the AI-native data storage capabilities and challenges in data preparation for CarbonData. --- README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f138497305..d2f3c8a52e 100644 --- a/README.md +++ b/README.md @@ -17,8 +17,16 @@ <img src="/docs/images/CarbonData_logo.png" width="200" height="40"> -Apache CarbonData is an indexed columnar data store solution for fast analytics on big data platform, e.g. Apache Hadoop, Apache Spark, etc. +- AI-native data storage for CarbonData's new scope, In AI projects, data scientists and engineers spend 80% of their time on data preparation. Traditional storage presents numerous bottlenecks in this process: + - Data silos: Training data may be scattered across data lakes, data warehouses, file systems, object storage, and other locations, making integration difficult. + - Performance bottlenecks:Training phase: High-speed, low-latency data throughput is required to feed GPUs to avoid expensive GPU resources sitting idle. + - Inference phase: High-concurrency, low-latency vector similarity search capabilities are required. + - Complex data formats: AI processes data types far beyond tables, including unstructured data (images, videos, text, audio) and semi-structured data (JSON, XML). Traditional databases have limited capabilities for processing and querying such data. + - Lack of metadata management: The lack of effective management of rich metadata such as data versions, lineage, annotation information, and experimental parameters leads to poor experimental reproducibility. + - Vectorization requirements: Modern AI models (such as large language models) convert all data into vector embeddings. Traditional storage cannot efficiently store and retrieve high-dimensional vectors. + +- In the previous releases Apache CarbonData is an indexed columnar data store solution for fast analytics on big data platform, e.g. Apache Hadoop, Apache Spark, etc. You can find the latest CarbonData document and learn more at: [https://carbondata.apache.org](https://carbondata.apache.org/)
