This is an automated email from the ASF dual-hosted git repository.
chenliang613 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git
The following commit(s) were added to refs/heads/master by this push:
new a7b6dffd4f add AI NATIVE DATA FOLDER
a7b6dffd4f is described below
commit a7b6dffd4f1dbbf1304b73be86d9706938563814
Author: chenliang613 <[email protected]>
AuthorDate: Mon Oct 6 20:22:41 2025 +0800
add AI NATIVE DATA FOLDER
---
.DS_Store | Bin 6148 -> 6148 bytes
AI-DATA/README.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 46 insertions(+)
diff --git a/.DS_Store b/.DS_Store
index f1282cea4f..076251c989 100644
Binary files a/.DS_Store and b/.DS_Store differ
diff --git a/AI-DATA/README.md b/AI-DATA/README.md
new file mode 100644
index 0000000000..ef40ce0a33
--- /dev/null
+++ b/AI-DATA/README.md
@@ -0,0 +1,46 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to you under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<img src="/docs/images/CarbonData_logo.png" width="200" height="40">
+
+
+## What is AI-native data storage
+
+* AI-native data storage is a data storage and management system designed and
built specifically for the needs of artificial intelligence (AI) workloads,
particularly machine learning and deep learning. Its core concept is to
transform data storage from a passive, isolated component of the AI process
into an active, intelligent, and deeply integrated infrastructure.
+
+## Why AI-native data storage for CarbonData's new scope
+
+In AI projects, data scientists and engineers spend 80% of their time on data
preparation. Traditional storage presents numerous bottlenecks in this process:
+
+Data silos: Training data may be scattered across data lakes, data warehouses,
file systems, object storage, and other locations, making integration difficult.
+
+Performance bottlenecks:
+
+Training phase: High-speed, low-latency data throughput is required to feed
GPUs to avoid expensive GPU resources sitting idle.
+
+Inference phase: High-concurrency, low-latency vector similarity search
capabilities are required.
+
+Complex data formats: AI processes data types far beyond tables, including
unstructured data (images, videos, text, audio) and semi-structured data (JSON,
XML). Traditional databases have limited capabilities for processing and
querying such data.
+
+Lack of metadata management: The lack of effective management of rich metadata
such as data versions, lineage, annotation information, and experimental
parameters leads to poor experimental reproducibility.
+
+Vectorization requirements: Modern AI models (such as large language models)
convert all data into vector embeddings. Traditional storage cannot efficiently
store and retrieve high-dimensional vectors.
+
+
+## About
+Apache CarbonData is an open source project of The Apache Software Foundation
(ASF).
+
