kbendick commented on a change in pull request #4073:
URL: https://github.com/apache/iceberg/pull/4073#discussion_r816483023
##########
File path: core/src/main/java/org/apache/iceberg/CatalogProperties.java
##########
@@ -81,4 +90,54 @@ private CatalogProperties() {
public static final String APP_ID = "app-id";
public static final String USER = "user";
+ /**
+ * Listeners are registered using catalog properties following the pattern of
+ * listeners.(listener-name).(listener-property)=(property-value)
+ * <p>
+ * A listener name cannot contain dot (.) character
+ * The specified listener is registered when a catalog is initialized
+ * <p>
+ * For example, there is the set of catalog properties registering an AWS
SQS listener of name prod:
+ * <ul>
+ * <li>listener.prod.impl=org.apache.iceberg.aws.sns.SnsListener
+ * <li>listener.prod.event-types=scan,incremental-scan
+ *
<li>listener.prod.sns.topic-arn=arn:aws:sns:us-east-2:123456789012:MyTopic
+ * </ul>
+ */
+ public static String listenerCatalogProperty(String listenerName, String
listenerProperty) {
+ return "listener." + listenerName + "." + listenerProperty;
+ }
+
+ /**
+ * Parse the listener name and listener property from a catalog property
string
+ * @param listenerCatalogProperty listener catalog property
+ * @return a pair of the listener name and listener property
+ */
+ public static Optional<Pair<String, String>>
parseListenerCatalogProperty(String listenerCatalogProperty) {
+ Matcher matcher =
Pattern.compile("^listener[.](?<name>[^\\.]+)[.](?<property>.+)$")
+ .matcher(listenerCatalogProperty);
+ if (matcher.matches()) {
+ return Optional.of(Pair.of(matcher.group("name"),
matcher.group("property")));
+ }
+
+ return Optional.empty();
+ }
+
+ /**
+ * Listener property describing the implementation Java class name of the
listener for dynamic loading
+ */
+ public static final String LISTENER_PROPERTY_IMPL = "impl";
+
+ /**
+ * Listener property describing the event types that a listener subscribes
to.
+ * The value is a comma delimited list of event types (Java class name),
+ * e.g.
org.apache.iceberg.events.ScanEvent,org.apache.iceberg.events.IncrementalScanEvent.
+ * If not specified, the listener subscribes to events listed in {@link
#LISTENER_EVENT_TYPES_DEFAULT}
+ */
+ public static final String LISTENER_PROPERTY_EVENT_TYPES = "event-types";
+ public static final Set<Class<?>> LISTENER_EVENT_TYPES_DEFAULT =
ImmutableSet.of(
+ ScanEvent.class,
+ IncrementalScanEvent.class,
+ CreateSnapshotEvent.class
Review comment:
We might want to review these events, and ensure that they are fired in
all of the situations that would be expected for a pipeline like this. Given
that we're somewhat repurposing them from WAP if I'm not mistaken.
Though I might very much be mistaken. but we might want to look into
`IncrementalScanEvent`. I think it fires in some situations that might not be
exactly relevant (in metadata tables) but it's been a while since I checked so
I could be off.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
