vernedeng commented on code in PR #6275:
URL: https://github.com/apache/inlong/pull/6275#discussion_r1005157552
##########
inlong-manager/manager-dao/src/main/resources/mappers/StreamSinkEntityMapper.xml:
##########
@@ -350,6 +350,13 @@
</if>
</where>
</select>
+ <select id="selectAllStreamSinks"
resultType="org.apache.inlong.manager.dao.entity.StreamSinkEntity">
+ select
+ <include refid="Base_Column_List"/>
+ from stream_sink
+ where is_deleted = 0
+ group by inlong_cluster_name, sort_task_name, sort_consumer_group,
sink_type, data_node_name
Review Comment:
The way sort-standalone manager metadata is to prepare all configs and cache
them in local memory, then reload them in a certain time interval.
Because the number of sort-standalone node is more than ten thousand, it
will introduce very high load if prepare each node config after each request.
The cpu and memory utility is less than 5% and 10% of a 8 cores, 20GB
manager node, in the senarios of more than 10 thousand groupId, 20 thousand
streamsink and 1 hundred dataNodes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
