[
https://issues.apache.org/jira/browse/COMDEV-506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhimin Li updated COMDEV-506:
-----------------------------
Remaining Estimate: 200h (was: 2,016h)
Original Estimate: 200h (was: 2,016h)
> [GSoC] RocketMQ TieredStore Integration with HDFS
> -------------------------------------------------
>
> Key: COMDEV-506
> URL: https://issues.apache.org/jira/browse/COMDEV-506
> Project: Community Development
> Issue Type: New Feature
> Environment: rocketmq,hdfs
> Reporter: Zhimin Li
> Priority: Major
> Labels: HDFS, RocketMQ, gsoc2023
> Original Estimate: 200h
> Remaining Estimate: 200h
>
> h2. [GSoC] RocketMQ TieredStore Integration with HDFS
> Github Issue: [https://github.com/apache/rocketmq/issues/6282]
> h3. Apache RocketMQ and HDFS
> * Apache RocketMQ is a cloud native messaging and streaming platform,
> making it simple to build event-driven applications.
> * Hadoop Distributed File System (HDFS) is a distributed file system
> designed to store and manage large data sets across multiple servers or
> clusters. HDFS provides a reliable, scalable, and fault-tolerant platform for
> storing and accessing data that can be accessed by a variety of applications
> running on the hadoop cluster.
> h3. Background
> High-speed storage media, such as solid-state drives (SSDs), are typically
> more expensive than traditional hard disk drives (HDDs). To minimize storage
> costs, the local data disk size of a rocketmq broker is often limited. HDFS
> can store large amounts of data at a lower cost, it has better support for
> storing and retrieving data sequentially rather than randomly. In order to
> preserve message data over a long period or facilitate message export, the
> RocketMQ project previously introduced a tiered storage plugin. Now it is
> necessary to implement a storage plugin to save data on hdfs.
> h3. Relevant Skills
> * Interest in messging middleware and distributed storage system
> * Java development skills
> * Having a good understanding of rocketmq and hdfs models
> Anyways, the most important relevant skill is motivation and readiness to
> learn during the project!
> h3. Tasks
> * understand the basic concepts and principles in distributed systems
> * provide related design documents
> * develop one that uses hdfs as the backend storage plugin to store rocketmq
> message data
> * write effective unit test code
> * *suggest improvements to the tiered storage interface
> * *what ever comes in your mind further ideas are always welcome
> h3. Learning Material
> * RocketMQ HomePage
> ([https://rocketmq.apache.org|https://rocketmq.apache.org/]) *Github*:
> [https://github.com/apache/rocketmq]
> * RocketMQ Tiered Storage Design
> ([https://github.com/apache/rocketmq/wiki/RIP-57-Tiered-storage-for-RocketMQ])
> * HDFS HomePage
> ([https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html])
> h3. Name and contact information
> * Mentor: Zhimin Li, Apache RocketMQ Committer,
> [[email protected]|mailto:[email protected]]
> * Mailing List: [[email protected]|mailto:[email protected]]
> * Website: [https://rocketmq.apache.org/] and [https://hadoop.apache.org/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
