chessplay commented on PR #7361:
URL: https://github.com/apache/seatunnel/pull/7361#issuecomment-2324335149
I am working on a streaming job using this feature, where I need to parse a
Kafka topic containing a protobuf message with nested objects in an array.
Below is the relevant portion of my configuration:
hocon
复制代码
source {
Kafka {
schema = {
fields {
sn = string
tenantId = int
groupId = int
utcCode = int
updateTime = bigint
totalUsers = int
array{CurrentUserInfo {
userId = int
mac = string
userIp = string
rssi = int
uplinkRate = float
upRate = bigint
timeDelay = int
score = int
}
}
}
}
topic = "pb-wirelessUser"
format = protobuf
protobuf_message_name = CurrentUserContent
protobuf_schema = """
syntax = "proto3";
package com.ruijie.cloud.macc.log.proto.sta;
message CurrentUserContent {
string sn = 1;
uint32 tenantId = 2;
uint32 groupId = 4;
int32 utcCode = 8;
int64 updateTime = 9;
uint32 totalUsers = 11;
repeated CurrentUserInfo userList = 20;
}
message CurrentUserInfo {
int32 userId = 1;
string mac = 2;
string userIp = 3;
int32 rssi = 7;
float uplinkRate = 8;
int64 upRate = 11;
int32 timeDelay = 17;
int32 score = 25;
}
"""
consumer.group = "seatunnel-test"
bootstrap.servers = "192.168.105.102:9092"
result_table_name = "kafka_table"
}
}
transform {
Sql {
source_table_name = "kafka_table"
result_table_name = "kafka_table2"
query = "select CurrentUserInfo from kafka_table"
}
}
sink {
Console {
source_table_name = "kafka_table2"
}
}
The protobuf message has a repeated field CurrentUserInfo as part of the
CurrentUserContent message. My question is: How should I configure SeaTunnel to
correctly parse and extract the nested CurrentUserInfo array?
I've tried the above configuration, but I'm not sure if this is the correct
approach. Any guidance or examples of similar configurations would be greatly
appreciated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
