This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 4e46ce7d096f [docs](update) Update Docs Migrate to OLAP (#510)
4e46ce7d096f is described below
commit 4e46ce7d096f825ccb37e5d38cbb9dcd3eaee110
Author: KassieZ <[email protected]>
AuthorDate: Fri Apr 5 15:42:53 2024 +0800
[docs](update) Update Docs Migrate to OLAP (#510)
---
static/images/auto-table-creation.jpeg | Bin 0 -> 361055 bytes
static/images/high-speed-stability.jpeg | Bin 0 -> 424060 bytes
static/images/x2doris.jpg | Bin 0 -> 1411492 bytes
.../import/migrate-data-from-other-olap.md | 40 ++++++++++++++++++++-
4 files changed, 39 insertions(+), 1 deletion(-)
diff --git a/static/images/auto-table-creation.jpeg
b/static/images/auto-table-creation.jpeg
new file mode 100644
index 000000000000..b65474623516
Binary files /dev/null and b/static/images/auto-table-creation.jpeg differ
diff --git a/static/images/high-speed-stability.jpeg
b/static/images/high-speed-stability.jpeg
new file mode 100644
index 000000000000..cbf79c4f13d1
Binary files /dev/null and b/static/images/high-speed-stability.jpeg differ
diff --git a/static/images/x2doris.jpg b/static/images/x2doris.jpg
new file mode 100644
index 000000000000..9f5f1b0b4753
Binary files /dev/null and b/static/images/x2doris.jpg differ
diff --git
a/versioned_docs/version-2.0/data-operate/import/migrate-data-from-other-olap.md
b/versioned_docs/version-2.0/data-operate/import/migrate-data-from-other-olap.md
index 2111d10ebcb8..9b5dda19599b 100644
---
a/versioned_docs/version-2.0/data-operate/import/migrate-data-from-other-olap.md
+++
b/versioned_docs/version-2.0/data-operate/import/migrate-data-from-other-olap.md
@@ -24,4 +24,42 @@ specific language governing permissions and limitations
under the License.
-->
-Coming soon
\ No newline at end of file
+To migrate data from other OLAP systems to Doris, you have a variety of
options:
+
+- For systems like Hive/Iceberg/Hudi, you can leverage Multi-Catalog to map
them as external tables and then use "Insert Into" to import the data into
Doris.
+
+- You can export data from the OLAP system into formats like CSV, and then
import the data files into Doris.
+
+- You can also leverage the connectors of the OLAP systems, use tools like
Spark / Flink, and then call the corresponding Doris Connector to write data
into Doris.
+
+In addition to the above methods, [VeloDB](https://www.velodb.io), the
commercial supporter of Apache Doris, provides a free visual data migration
tool: X2Doris. Developed by VeloDB, X2Doris is designed to migrate various
offline data into Apache Doris. It combines the funtionalities of automatic
table creation and data migration. Currently, it supports migrating data to
Doris from databases including Apache Doris/Hive/Kudu and StarRocks. The entire
process is performed through a visual [...]
+
+## X2Doris
+
+### Support multiple data sources
+
+As a one-stop data migration tool, X2Doris supports Apache Hive, Apache Kudu,
ClickHouse, StarRocks, and Apache Doris itself as data source. What's more,
there are more data sources such as Greenplum and Druid that are under
development and will be released subsequently. Among them, the Hive version
already supports Hive 1.x and 2.x, while Doris, StarRocks, Kudu, and other data
sources also support multiple different versions.
+
+Now, X2Doris is supported migrating data to Apache Doris and SelectDB,
including SelectDB Cloud and SelectDB Enterprise. With X2Doris, users can build
a complete database migration link from other OLAP systems to Apache Doris, and
can also achieve data backup and recovery between different Doris clusters.
+
+
+
+### Auto table creation
+
+One of the biggest challenges in data migration is how to create corresponding
target tables in Apache Doris for the source tables that need to be migrated.
In real business scenarios, there are often thousands of tables stored in Hive
or ClickHouse, and it would be extremely inefficient and impractical for users
to manually create tables and convert corresponding DDL statements.
+
+X2Doris has been adapted for this scenario. Taking Hive table migration as an
example, when migrating Hive tables, X2Doris automatically creates Duplicate
Key model tables (which can also be manually modified) in Apache Doris and
reads the metadata information of Hive tables. It automatically identifies
partition fields based on field names and types, and if partitions are
detected, it prompts for partition mapping. Finally, it directly generates the
corresponding Doris target table DDL.
+
+When the upstream data source is Doris/StarRocks, X2Doris automatically parses
the table model based on the source table information, maps the source table
field types to the corresponding target field types, and identifies and
processes upstream properties parameters, converting them into attribute
parameters for the corresponding target table. In addition, X2Doris has also
enhanced support for complex types, enabling data migration for Array, Map, and
Bitmap types.
+
+
+
+### High speed & stability
+
+For data writing, X2Doris has specifically optimized the reading process. By
optimizing the data batching logic, it further reduces memory usage.
Additionally, significant improvements and enhancements have been made to
Stream Load write requests, optimizing memory usage and release, further
enhancing the speed and stability of data migration.
+
+Compared to other similar migration tools, X2Doris offers a performance
advantage of approximately 2-10 times. For example, when using a single machine
with 1G of memory, other tools take approximately 90 seconds to synchronize 50
million rows of data in full, while X2Doris completes the task in less than 50
seconds, achieving a nearly 100% performance improvement.
+
+In a practical large-scale log data migration scenario, with individual data
records averaging 1KB in size, a single table containing nearly 100 million
records, and a total storage space of approximately 90 GB, X2Doris can complete
the full table migration in just 2 minutes, with an average write speed of
nearly 800 MB/s.
+
+
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
