Niranjan Shukla <[email protected]> wrote: > discover hubs (hubs full of such useful information > derived from message content filtering), a potential > publisher would come to know well about all the > trending topics at a given hub
I believe that what you are describing may be addressed by the concept of "Advertisements" which is well-known within the realm of PubSub systems, but not currently implemented in PubSubHubbub. Take a look at an ancient Internet Draft that I wrote many years ago (2002) and consider if the "well-known Topics Topic" that is described there might be something like what you want. See: http://tools.ietf.org/html/draft-wyman-apex-pubsub-00#page-32 Advertisements, if implemented in PubSubHubbub, would allow a hub to "advertise" to others (subscribers or hubs) information about the topics that it is serving. The simplest way to do this would be to have a hub publish a message to a well-known topic whenever it begins or ends service for a particular feed. A hub might also periodically republish those topics that it continues to serve (perhaps once a month?). Hubs would then subscribe to other hubs' "topic.topic" to discover what topics those hubs were handling. Ideally, hubs would then rebroadcast on their own topic.topics the advertisements that they had received from others. (Of course, a hub would keep track of externally generated advertisements and would never publish any single advertisement more than once during some time period (perhaps one month?). Fingerprints would work well here.) The result would be a graph of hubs through which one could efficiently flood messages describing new feeds or feeds that are being dropped. A mesh of hubs that communicated via such advertisements would iterate towards a shared understanding of what feeds were available and which hubs were supporting them. Such a system would make it vastly easier to construct services that rely on being aware of a high percentage of available feeds. It would make topic/feed discovery massively easier and thus reduce the barriers to entry for new services. (But, that reduction in the barriers to entry is, unfortunately, why at least some folk would not like such a system -- they like the proprietary advantage that comes from being an early market entrant.) Note: I'm sure that some folk will immediately recognize the similarity between the "flooding" model discussed above and that which is used with NNTP. Well, NNTP may be an old protocol, but it had its good points... Of course, advertisements need not be limited only to statements about the names of feeds/topics that are being served -- this is especially the case if content-based pubsub becomes supported in hubs that allow filtering against aggregate or fire-hose topics. One might, for instance, have hubs advertise the "kind" and quantity of content that they serve. Thus, a hub might announce that it tends to aggregate "space weather" data or social data or microblogs or French language content or RDF data related to genetics, etc. and thus provide content-based subscribers with clues as to which hubs would serve their particular content-based subscription needs. Adding advertisement support to PSHB would make the existing spec more complex -- which would be unpopular... However, this is also the sort of thing that could easily be addressed in a distinct specification that would be an add-on to the base. bob wyman On Thu, Aug 26, 2010 at 2:28 PM, Niranjan Shukla <[email protected]>wrote: > Hi, > > What I was implying by point 1 was this - Yes it was message content > filtering as Bob said but additionally, some useful information could be > directed back to the publishers and other potential publishers which would > help them reach out to a yet wider set of subscribers. Furthermore, point 3 > conveyed that if it were logical to discover hubs (hubs full of such useful > information derived from message content filtering), a potential publisher > would come to know well about all the trending topics at a given hub (from > my limited experience on enterprise integration, i understand that > publishers and subscribers generally agree upon topics but on the web, you > may have a to-be publisher wanting to know what are the trending topics). > Lastly, a Hub could convey topic trends in its vicinity to networked hubs. > > I do not intend to touch the spec. in any way, but if someone feels that a > publisher may gain positively from knowing the topics-in-trend at a given > hub or a networked hub, please let me know your thoughts. This is especially > possible considering shared logical hubs. (I'm glad point 2. was not taken > further). > Best regards, > Niranjan > ---------------------------------- > > On Tue, Aug 24, 2010 at 11:15 PM, Bob Wyman <[email protected]> wrote: > >> On Aug 23, 1:00 pm, Niranjan <[email protected]> wrote: >> > 1. Can Hubs be engineered to understand the message-content they >> > handle? >> PSHB as currently defined is a "topic-based" PubSub system. In PSHB, >> you subscribe to a stream of data that is published to a particular >> "topic" or within a named feed. It appears that what you're asking for >> is, in part, for PSHB to be extended to also support "content-based" >> PubSub -- or, the ability to subscribe to filtered streams of data >> where the filtering is performed on the content of the message, not >> merely on the name of the topic to which the data was published. Given >> that PSHB is already topic-based, my assumption is that you would want >> content-based filtering within streams of data which would correspond >> to the existing Topics. Thus, you're asking for a hybrid system that >> implements content-based filtering over topics. I recently wrote a >> note proposing precisely this. Take a look the following and consider >> providing your thoughts: >> >> http://groups.google.com/group/pubsubhubbub/browse_thread/thread/2fdbd65bbb8602c8# >> >> bob wyman >> > >
