[
https://issues.apache.org/jira/browse/BOOKKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405698#comment-13405698
]
Sijie Guo commented on BOOKKEEPER-78:
-------------------------------------
> Does that mean filtering is only in protocol-specific clients, or would there
> be a filtering mechanism in the broker itself?
>From our works in BOOKKEEPER-321, we would like to provide a generic filtering
>mechanism in hedwig. The message filter could be run either in server-side or
>client-side.
The message filter interface extends existing MessageFilter interface as below.
{code}
public interface MessageFilter {
/**
* Initialize the message filter with subscription preferences.
*
* @param preferences
* Preferences of a subscriber
*/
public void initialize(SubscriptionPreferences preferences);
/**
* Tests whether a particular message passes the filter or not
*
* @param message
* Message try to be delivered to the subscriber
* @return whether the message should be delivered to subscriber or not.
*/
public boolean testMessage(Message message);
}
{code}
For *Messages*, there is a header part in each of them. So
applications/protocols uses Hedwig could put their metadata in the header of
the messages for filtering.
*SubscriptionPreferences* is also a kind of header part recorded in
subscription metadata. So applications/protocols uses Hedwig could put their
specific subscription metadata in it. For example, JMS provider might put its
SQL-like filtering expression in it for different subscribers.
The message filter could be run either in server-side or client-side.
for client-side, user could attach a message filter when startDelivery to
receive messages as below. the message is delivered to messageHandler only when
it passed messageFilter.
{code}
public void startDelivery(ByteString topic, ByteString subscriberId,
MessageHandler messageHandler, MessageFilter
messageFilter)
throws ClientNotSubscribedException, AlreadyStartDeliveryException;
{code}
for server-side, the subscriber could specify a server-side running message
filter (its class name) in its subscription preferences. The message filter
would be loaded dynamically.
{code}
// record all preferences for a subscription,
// would be serialized to be stored in meta store
message SubscriptionPreferences {
// user customized subscription options
optional Map options = 1;
///
/// system defined options
///
// message bound
optional uint32 messageBound = 2;
// server-side message filter
optional string messageFilter = 3;
}
{code}
for JMS provider, it could leverage the filter plugin mechanism provided by
hedwig to implement its SQL-like filtering logic.
for other hedwig application, they could use raw filter plugin mechanism to
filter the messages in their ways.
> Don't you want to have typed SQL-like filtering capabilities by default?
As described above, hedwig just provides filter plugin mechanism to make it
light-weight and flexible. SQL-like filtering would be an filter plugin
provided. If possible, SQL-like filter would be a common plugin not be bound
too tight with JMS Spec. so JMS provider could use the SQL-like filter while
other people might reuse it too.
> filterable metadata fields in Hedwig's message definition
> ---------------------------------------------------------
>
> Key: BOOKKEEPER-78
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-78
> Project: Bookkeeper
> Issue Type: New Feature
> Components: hedwig-client, hedwig-server
> Reporter: Matthieu Morel
>
> In order to efficiently implement filtering of Hedwig messages, Hedwig should
> be able to rely on metadata information. (i.e. without needing to deserialize
> the content of the message)
> Filtering could use a subset of SQL (like in the JMS spec), leading to
> queries such as :
> "header1 like 'a' AND header2 IS NOT NULL"
> For that purpose, I propose to add customizable metadata to the definition of
> Hedwig messages, as header fields.
> Metadata must be customizable because it may be arbitrary. We should provide
> "map-like" containers according to the type of the metadata field. Metadata
> fields would be accessed by name.
> There are predefined headers for JMS that could be added as metadata fields
> such as : destination (~topic), delivery mode (persistent or not),
> expiration, priority, timestamp, correlation id (link to other message),
> reply to, type and redelivered. I think only a subset of these should be
> predefined headers, if any.
> Adding metadata fields to Hedwig messages implies modifying the message
> definition, which does not break backward compatibility when those fields are
> added as optional in the protocol buffer definition.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
