duheng created COMDEV-360:
-----------------------------
Summary: RocketMQ Connect Hive
Key: COMDEV-360
URL: https://issues.apache.org/jira/browse/COMDEV-360
Project: Community Development
Issue Type: Wish
Components: GSoC/Mentoring ideas
Reporter: duheng
h3. Content
The Hive sink connector allows you to export data from Apache RocketMQ topics
to HDFS files in a variety of formats and integrates with Hive to make data
immediately available for querying with HiveQL. The connector periodically
polls data from RocketMQ and writes them to HDFS.
The data from each RocketMQ topic is partitioned by the provided partitioner
and divided into chunks. Each chunk of data is represented as an HDFS file with
topic, queueName, start and end offsets of this data chunk in the filename.
So, in this project, you need to implement a Hive sink connector based on
OpenMessaging connect API, and run it on RocketMQ connect runtime.
You should learn before applying for this topic
Hive/Apache RocketMQ/Apache RocketMQ Connect/ OpenMessaging Connect API
h3. Mentor
[[email protected]|mailto:[email protected]],
[[email protected]|mailto:[email protected]]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
