preface
The core of eventmesh is based on storage components with publish and subscribe
functions. In order to support more scenarios, storage components without
publish and subscribe functions need to be supported.
introduce
Storage connector is a link connecting storage in the event connector system.
Other connections are
pubsub-connector
function-connector
stream-connector
notify-connector
service-connector
workflow-connector
Design
From the above flow chart, we find that there are two modes 1. The native
subscription and publishing mode of eventmesh 2. Data is the business data of
the business system. We hope to distribute the business data through eventmesh.
Storage interface
public interface StorageOperation {
/** * Initialize storage operation */ public void init(Properties properties)
throws Exception; /** * insert event */ public int insertEvent(CloudEvent
cloudEvent); /** * Main users mark events */ public int updateEvent(CloudEvent
cloudEvent); /** * delete event */ public int deleteEvent(CloudEvent
cloudEvent); /** * select event */ public CloudEvent queryEvent(CloudEvent
cloudEvent); /** * select event list */ public List<CloudEvent>
queryEventList(CloudEvent cloudEvent,int pageNum,int pageSize); /** * Get
topic, consumer group and other information */ public Object queryMetaData();
/** * To create topic, some storage must be created first * For example, create
tables in relational databases */ public int createTopic(CloudEvent
cloudEvent); /** *Create a consumer group, which is mainly used in the multi
consumer group mode */ public int createConsumerGroup(CloudEvent cloudEvent);
} `
Simulate push operation
At present, common databases do not support the push mechanism, so the
internal implementation of eventmesh is required to actively obtain data to
simulate push
keynote
Metadata management
Routing load
JDBC-storage-connection-design
keynote
SQL compatible. Common relational databases include mysql, service SQL, DB2,
PG, and Oracle. The SQL of each database may be different
Create table statement
Paging statement
Insert, select statement
table design
sql create table cloud_event( cloud_event_id varchar(32) not null comment
'Unique construction', business_id varchar(64) not null comment 'Business
ID', producer_group varchar(32) not null comment 'Producer group',
event_data varchar(2048) not null comment 'event data', event_tag
varchar(16) not null default '' comment 'event tag', event_extensions
varchar(2048) not null default '' comment ' event extensions', send_time
datetime not null default current_timestamp comment 'send time',
consumer_group varchar(32) not null comment 'consumer group', event_state
int not null default 1 comment '1: create 2:Receiving 3:In consumption
4:success 5:Consumption failure', update_time datetime not null default
current_timestamp on update current_timestamp comment 'update time', );
difficulty
Consumption site management
There are two situations
The first one is a consumer
Second, multiple consumers
One consumer is relatively easy to handle, if maintaining multiple consumers
metadata management
How to manage metadata among multiple storage nodes. For example, there
is only node a at the beginning, with 1, 2, 3, and 4 topics. Then add node B.
If you synchronize topic in node B
Timing message
request message
anyc message
Publish cannot be truly asynchronous. Relational databases are semi-finished
