Re: [Services] EventStreams launch and RCStream deprecation

[Andrew Otto]

Alright, we’ve done it!

RCStream is disabled, so any remaining socket.io service connecting to
stream.wikimedia.org/rc will fail.

Thanks all!


On Thu, Jun 22, 2017 at 1:00 PM, Andrew Otto  wrote:

> Hi all,
>
> This is just a friendly reminder that we plan to turn off the RCStream
> service after July 7th.
>
> We’re tracking as best we can the progress of porting clients over at
> https://phabricator.wikimedia.org/T156919.  But, we can only help with
> what we know about.  If you’ve got something still running on RCStream that
> hasn’t yet ported, let us know, and/or switch soon!
>
> Thanks!
> -Andrew Otto
>
>
>
> On Wed, Feb 8, 2017 at 9:28 AM, Andrew Otto  wrote:
>
>> Hi everyone!
>>
>> Wikimedia is releasing a new service today: EventStreams
>> .  This service allows
>> us to publish arbitrary streams of JSON event data to the public.
>> Initially, the only stream available will be good ol’ RecentChanges
>> .  This event stream
>> overlaps functionality already provided by irc.wikimedia.org and RCStream
>> .  However, this new
>> service has advantages over these (now deprecated) services.
>>
>>
>>1.
>>
>>We can expose more than just RecentChanges.
>>2.
>>
>>Events are delivered over streaming HTTP (chunked transfer) instead
>>of IRC or socket.io.  This requires less client side code and fewer
>>special routing cases on the server side.
>>3.
>>
>>Streams can be resumed from the past.  By using EventSource, a
>>disconnected client will automatically resume the stream from where it 
>> left
>>off, as long as it resumes within one week.  In the future, we would like
>>to allow users to specify historical timestamps from which they would like
>>to begin consuming, if this proves safe and tractable.
>>
>>
>> I did say deprecated!  Okay okay, we may never be able to fully deprecate
>> irc.wikimedia.org.  It’s used by too many (probably sentient by now)
>> bots out there.  We do plan to obsolete RCStream, and to turn it off in a
>> reasonable amount of time.  The deadline iis July 7th, 2017.  All
>> services that rely on RCStream should migrate to the HTTP based
>> EventStreams service by this date.  We are committed to assisting you in
>> this transition, so let us know how we can help.
>>
>> Unfortunately, unlike RCStream, EventStreams does not have server side
>> event filtering (e.g. by wiki) quite yet.  How and if this should be done
>> is still under discussion .
>>
>> The RecentChanges data you are used to remains the same, and is available
>> at https://stream.wikimedia.org/v2/stream/recentchange. However, we may
>> have something different for you, if you find it useful. We have been
>> internally producing new Mediawiki specific events
>> 
>> for a while now, and could expose these via EventStreams as well.
>>
>> Take a look at these events, and tell us what you think.  Would you find
>> them useful?  How would you like to subscribe to them?  Individually as
>> separate streams, or would you like to be able to compose multiple event
>> types into a single stream via an API?  These things are all possible.
>>
>> I asked for a lot of feedback in the above paragraphs.  Let’s try and
>> centralize this discussion over on the mediawiki.org EventStreams talk
>> page .   In summary,
>> the questions are:
>>
>>
>>-
>>
>>What RCStream clients do you maintain, and how can we help you
>>migrate to EventStreams?
>>
>>-
>>
>>Is server side filtering, by wiki or arbitrary event field, useful to
>>you? 
>>-
>>
>>Would you like to consume streams other than RecentChanges?
>>  (Currently
>>available events are described here
>>
>> 
>>.)
>>
>>
>>
>> Thanks!
>> - Andrew Otto
>>
>>
>>
>
___
Services mailing list
Services@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/services

Re: [Services] EventStreams launch and RCStream deprecation

[Andrew Otto]

Hi all,

This is just a friendly reminder that we plan to turn off the RCStream
service after July 7th.

We’re tracking as best we can the progress of porting clients over at
https://phabricator.wikimedia.org/T156919.  But, we can only help with what
we know about.  If you’ve got something still running on RCStream that
hasn’t yet ported, let us know, and/or switch soon!

Thanks!
-Andrew Otto



On Wed, Feb 8, 2017 at 9:28 AM, Andrew Otto  wrote:

> Hi everyone!
>
> Wikimedia is releasing a new service today: EventStreams
> .  This service allows
> us to publish arbitrary streams of JSON event data to the public.
> Initially, the only stream available will be good ol’ RecentChanges
> .  This event stream
> overlaps functionality already provided by irc.wikimedia.org and RCStream
> .  However, this new
> service has advantages over these (now deprecated) services.
>
>
>1.
>
>We can expose more than just RecentChanges.
>2.
>
>Events are delivered over streaming HTTP (chunked transfer) instead of
>IRC or socket.io.  This requires less client side code and fewer
>special routing cases on the server side.
>3.
>
>Streams can be resumed from the past.  By using EventSource, a
>disconnected client will automatically resume the stream from where it left
>off, as long as it resumes within one week.  In the future, we would like
>to allow users to specify historical timestamps from which they would like
>to begin consuming, if this proves safe and tractable.
>
>
> I did say deprecated!  Okay okay, we may never be able to fully deprecate
> irc.wikimedia.org.  It’s used by too many (probably sentient by now) bots
> out there.  We do plan to obsolete RCStream, and to turn it off in a
> reasonable amount of time.  The deadline iis July 7th, 2017.  All
> services that rely on RCStream should migrate to the HTTP based
> EventStreams service by this date.  We are committed to assisting you in
> this transition, so let us know how we can help.
>
> Unfortunately, unlike RCStream, EventStreams does not have server side
> event filtering (e.g. by wiki) quite yet.  How and if this should be done
> is still under discussion .
>
> The RecentChanges data you are used to remains the same, and is available
> at https://stream.wikimedia.org/v2/stream/recentchange. However, we may
> have something different for you, if you find it useful. We have been
> internally producing new Mediawiki specific events
> 
> for a while now, and could expose these via EventStreams as well.
>
> Take a look at these events, and tell us what you think.  Would you find
> them useful?  How would you like to subscribe to them?  Individually as
> separate streams, or would you like to be able to compose multiple event
> types into a single stream via an API?  These things are all possible.
>
> I asked for a lot of feedback in the above paragraphs.  Let’s try and
> centralize this discussion over on the mediawiki.org EventStreams talk
> page .   In summary,
> the questions are:
>
>
>-
>
>What RCStream clients do you maintain, and how can we help you migrate
>to EventStreams?
>
>-
>
>Is server side filtering, by wiki or arbitrary event field, useful to
>you? 
>-
>
>Would you like to consume streams other than RecentChanges?
>  (Currently
>available events are described here
>
> 
>.)
>
>
>
> Thanks!
> - Andrew Otto
>
>
>
___
Services mailing list
Services@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/services

[Services] EventStreams launch and RCStream deprecation

[Andrew Otto]

Hi everyone!

Wikimedia is releasing a new service today: EventStreams
.  This service allows us
to publish arbitrary streams of JSON event data to the public.  Initially,
the only stream available will be good ol’ RecentChanges
.  This event stream overlaps
functionality already provided by irc.wikimedia.org and RCStream
.  However, this new service
has advantages over these (now deprecated) services.


   1.

   We can expose more than just RecentChanges.
   2.

   Events are delivered over streaming HTTP (chunked transfer) instead of
   IRC or socket.io.  This requires less client side code and fewer special
   routing cases on the server side.
   3.

   Streams can be resumed from the past.  By using EventSource, a
   disconnected client will automatically resume the stream from where it left
   off, as long as it resumes within one week.  In the future, we would like
   to allow users to specify historical timestamps from which they would like
   to begin consuming, if this proves safe and tractable.


I did say deprecated!  Okay okay, we may never be able to fully deprecate
irc.wikimedia.org.  It’s used by too many (probably sentient by now) bots
out there.  We do plan to obsolete RCStream, and to turn it off in a
reasonable amount of time.  The deadline iis July 7th, 2017.  All
services that rely on RCStream should migrate to the HTTP based
EventStreams service by this date.  We are committed to assisting you in
this transition, so let us know how we can help.

Unfortunately, unlike RCStream, EventStreams does not have server side
event filtering (e.g. by wiki) quite yet.  How and if this should be done
is still under discussion .

The RecentChanges data you are used to remains the same, and is available
at https://stream.wikimedia.org/v2/stream/recentchange. However, we may
have something different for you, if you find it useful. We have been
internally producing new Mediawiki specific events

for a while now, and could expose these via EventStreams as well.

Take a look at these events, and tell us what you think.  Would you find
them useful?  How would you like to subscribe to them?  Individually as
separate streams, or would you like to be able to compose multiple event
types into a single stream via an API?  These things are all possible.

I asked for a lot of feedback in the above paragraphs.  Let’s try and
centralize this discussion over on the mediawiki.org EventStreams talk page
.   In summary, the
questions are:


   -

   What RCStream clients do you maintain, and how can we help you migrate
   to EventStreams? 
   -

   Is server side filtering, by wiki or arbitrary event field, useful to
   you? 
   -

   Would you like to consume streams other than RecentChanges?
     (Currently
   available events are described here
   

   .)



Thanks!
- Andrew Otto
___
Services mailing list
Services@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/services