Hey Ilya, Thanks for jumping into the discussion. Please see below.
On Thu, Nov 18, 2010 at 4:47 PM, igrigorik <[email protected]> wrote: > Julien, since I was one of those people objecting to this model.. Let > me clarify my point: > > There is certainly a need for protected RSS feeds: password protected, > obfuscated URL's, etc. For those, the publisher is maintaining an ACL > for who should and/or should not access those feeds. But, if the > publisher makes all of their feeds publicly available individually, > then it seems to me that a missing firehose feed is simply a technical > hurdle. > Yes, and no. Many publishers want to know what's happening with their content. When apps start crawling a lot of their content, they usually want to know why and what they should expect from it. There are multiple concerns, from spam-dexing to single analytics. For many websites it's actually the same : you can use them or even prorgamatically access them, but they will start blocking you if you are too agressive and will probably offer you a commercial service for higher levels. My whole point is that one (publisher or subscriber) should not expect anything which is not technical based on the fact that some content is available as PubSubHubbub. Actually this isn't different from what it was 2 years ago : you could already try to list all feeds from a service and poll them all. It's likely that they would have quickly asked for details or investigated on what you're trying to achieve. Take Buzz as an example.. At PostRank we had the "firehose" even prior > to the "real firehose" by actively crawling Google's sitemap (public), > identifying users with Buzz feeds, and subscribing to their individual > PSHB feeds. It was painful, and not very efficient, but it worked and > did not violate any TOS. In effect, it was an artificial barrier - > granted, there may be a reason why you might want to make it hard to > get at this data, but the point is, outsiders are still able to do it. > And publishers can still block them! In the superfeedr approach, we use the publisher callback for each and _every_ subscription made to the hub. The rules are up to the publishers. Some of them will accept blindly any subscription, some others will limit the number of subscriptions to a given host/domain... etc. In the case of Posterous, same logic applies. I could run a crawler, > go gather all the RSS feeds, and call it a day. I'm not violating any > TOS, as much as I'm jumping over a technical barrier. If anything, > this is flawed thinking on the part of the distributor of those RSS > feeds. > Yes you could. I'm not sure about the TOS part though. And no matter what, it's likely that they will block you when they start seeing a massive amount of subscriptions if they're not confident that you're doing a "fair-use" of their data. The same will apply if you start polling them too aggressively. In all the points above, Superfeedr doesn't get to say a word. We're just applying what each publisher wants. Of course, we advise them, but most of the time, they have their own strategies and have their ideas on what they want to achieve. As far as pricing goes. I'm not the one to decide on your business > model, but as a developer/user I do think that double-charging (both > the provider, and the consumer) is counter-productive. Most publishers > have the problem of _nobody giving a damn_ about their content, and > not the other way around (certainly something we see a lot of at > PostRank). Hence, lowering the barriers to distribution for those > publishers is one of the key ways they can address this. That's why > RSS feeds are so valuable in the first place. Most publishers have > realized at this point that full-content RSS feeds are the way to go - > as in, give it away, give it away for free, and if they like it, > they'll come for more. Hail mary? Perhaps. But that's how the world > works in the age of content abundance. > Oddly enough I agree with you :) The double-charging is an unfortunate consequence of the fact that pretty each publisher has their own approach. Some of them don't want to charge, some of them want to charge a fixed feed, some of them want to do something variable... etc. Rather than try to deal with diverging interests, we decided that we should take our part and then let the publisher do whatever they want. You pay them for the data, you pay us for the transport. It's not perfect, but it's better than nothing and we're working on making this better. However, I think it's unfair to assume that all feeds are equal! I believe we have done a lot to raise adoption of PubSubHubbub (while we don't even have any postrank callbacks for public hubs we host... ), and you can already access thousands of feeds thru us with PubSubHubbub for free. So, my point is: charge the publisher for the PSHB hub. You're > providing a service, you can charge them based on amount of people > consuming their content (in other words, align yourself with their > success metric). But, for "Joe the developer".. I'm not paying to > access a push RSS feed when I can get that RSS feed for free by > walking around your fake fence - it's silly. Yes, I get it, it's more > efficient, but still. > Again, you'll be surprised that I agre with you and that we're working on something in this direction. Unfortunately, a year ago, PubSubHubbub had a kind of a chicken and egg issue and nobody wanted to pay for hubs. I have still yet to be convinced that there is room for a product of hubs, specially when this is an open protocol, which has both an open source implementation and an open running hub. > What bothers me is that every hub you lock in at superfeedr, is a hub > I can't access for free. THIS IS WRONG! I'm writing this in bold because I can't accept that anyone would miss it. You can obviously get the content from Tumblr, Posterous, Gowalla, Typepad or any hub we host for free, and without a superfeedr account, exactly like you do for the Google hub! I know you and I know that you can't be ill-intentioned, so I think we have communication work to do. Just to be sure : THERE IS NO DIFFERENCE FOR SUBSCRIBERS BETWEEN A SUPERFEEDR HOSTED HUB AND THE GOOGLE HUB. > In reverse, this is why I am so excited about > Google's PSHB hub - they get it. Publishers want distribution, they're > aligning themselves with the publishers. > Again, I really hope there has been a massive misunderstanding. Cheers, Julien > > ig > > > On Nov 17, 6:13 pm, Julien Genestoux <[email protected]> > wrote: > > Hello everybody, > > > > Today, twice I've had talks with people who assumed that by default all > the > > data available via PubSubHubbub was expected to be free and accessible by > > any one for any purpose. > > I think this is a 'wrong' idea and it doesn't serve us very well, so I > wrote > > a blog post about it :http://blog.superfeedr.com/not-a-license/ > > > > I think it's awesome that so many services, like Buzz grant an almost > > unlimited and full access to all their data, but I think we (and maybe > > anyone advocating) should make sure that we do not give the idea that by > > implementing PubSubHubbub people give away their data and any rights > around > > it. > > > > Any feedback is much appreciated! Has anyone bumped into the same > > mis-understanding? or worse, had them? > > > > Julien >
