Re: [Dev] [Siddhi] non-matched or expired events in pattern query

[Grainier Perera]

Hi Jay,

Since you are maintaining a jobTable, you can use that to identify
completed and non-completed jobs in a given time frame. Refer to the below
snippet;

@Plan:name('ExecutionPlan')


> @Import('subStream:1.0.0')
> define stream subStream (pid int, sid int, status string);


> @Import('pubStream:1.0.0')
> define stream pubStream (pid int, status string);


> @Export('jobsNotCompletedInTime:1.0.0')
> define stream jobsNotCompletedInTime (pid int, sid int);


> @Export('jobsCompletedInTime:1.0.0')
> define stream jobsCompletedInTime (pid int, sid int);


> @Export('foo:1.0.0')
> define stream foo (pid int, sid int);


> @From(eventtable='rdbms', jdbc.url='jdbc:mysql://localhost:3306/pubsub',
> username='root', password='root', driver.name='com.mysql.jdbc.Driver',
> table.name='jobTable')
> define table jobTable(pid int, sid int);


> from pubStream as p join jobTable as t
> on p.pid == t.pid
> select p.pid, t.sid
> insert into allPSJobs;



-- this will emit an event after 10 sec once a publish event arrives

from pubStream#window.time(10 sec)
> select *
> insert expired events into *expiredPublisherEvents*;



-- if the pattern matches within 10 sec, that means the job completed
> within time

from every (a=allPSJobs) -> s=subStream[pid==a.pid and sid == a.sid]
> within 10 sec
> select s.pid, s.sid
> insert into *jobsCompletedInTime*;


> -- delete completed jobs from the table, then what's left in the table are
> the uncompleted jobs
> from jobsCompletedInTime
> delete jobTable
> on jobTable.pid == pid and jobTable.sid == sid;


> -- so if we join expiredPublisherEvents with the jobTable, the result
> will be the jobs that are not completed within time

from expiredPublisherEvents as ep join jobTable as t
> on ep.pid == t.pid
> select ep.pid, t.sid
> insert into *jobsNotCompletedInTime*;


> -- now delete those uncompleted jobs from the jobTable as well
> from jobsNotCompletedInTime
> delete jobTable
> on jobTable.pid == pid and jobTable.sid == sid;



Hope that helped;

Regards,
Grainier.


On Fri, Sep 29, 2017 at 11:17 AM, Jayesh Senjaliya 
wrote:

> Hi Gobinath,
>
> Thanks for the suggestion about absence pattern, but we wont be able to
> upgrade to Siddhi 4 anytime soon.
>
> I am basically at the point where I can get all the relevant (subscribe)
> events that happened during the given interval of first arrival of publish
> events.
>
> here AllPublisher = all registered publisher-subscriber events ( for each
> publisher event )
>
> from registeryTable as s join #publisher as p on p.pid == s.pid
> select p.pid, s.sid insert into AllPublisher;
>
>
> from every(a=AllPublisher) -> s=subscriber[pid == a.pid]
> within 1 sec
> select a.pid, s.sid
> insert into completed_jobs_in_1_sec;
>
> now, i need to find out those events from publisher (a) that dint had any
> match from s within 1 sec.
>
> which I was expecting to be available with " insert *expired events* into
> not_completed_jobs", but looks like expired events are only available when
> window is used. I am also looking at the code to see if i should add this.
>
> Thanks
> Jayesh
>
>
>
>
>
>
>
>
>
> On Wed, Sep 27, 2017 at 4:21 AM, Gobinath  wrote:
>
>> Hi,
>>
>> If you can use Siddhi 4 snapshot release, it can be done using the new
>> feature 'Absent Pattern' added to Siddhi 4. The query to detect the events
>> that do not match the condition within 10 seconds is given below:
>>
>> from every e1=publisher -> not subscriber[e1.pid == pid] for 10 sec
>> select e1.pid
>> insert into not_completed_jobs_in_time;
>>
>> The above query waits for 10 seconds from the arrival of every publisher
>> event and if there is no subscriber event with an id satisfying the
>> condition arrived within that waiting period, the id of the publisher event
>> will be inserted into the not_completed_jobs_in_time stream.
>>
>> I guess the official document for Siddhi 4 is under construction. So you
>> can find more details about absent pattern at [1]
>>
>> Still, Siddhi 4 is not production ready so I wonder whether you can use
>> this feature or not.
>>
>> [1] http://www.javahelps.com/2017/08/detect-absence-of-events-ws
>> o2-siddhi.html
>>
>>
>>
>>
>> On Tue, Sep 26, 2017 at 10:05 PM, Jayesh Senjaliya 
>> wrote:
>>
>>> Hi Grainier,
>>>
>>> ya, i came across that example page, but i think that does not work in
>>> my use-case which is as follow.
>>>
>>> i have a publish event followed by multiple subscribe event for the same
>>> publish job.
>>> now i want to catch if certain jobs (publish -> subscribe) has been
>>> finished with 10 sec.
>>> I have all the registered jobs in db table, which i use to gather all
>>> the required publish-subscribe job events.
>>>
>>> define table jobTable( pid string, sid string);
>>> define stream pubStream (pid int, status string);
>>> define stream subStream (pid int, sid int, status string);
>>>
>>> -- this will get all the publish-> subscribe jobs events as master 

Re: [Dev] SCIM 2.0 as default in IS 5.4.0

[Johann Nallathamby]

Any thoughts on the above idea or did we fix it in a different way?

On Sat, Sep 23, 2017 at 9:46 AM, Johann Nallathamby  wrote:

> See if this idea would work?
>
> We currently set a threadlocal variable inside the SCIM endpoint
> component, to identify the SCIM service provider inside some of our
> implementations. Can we improve this variable value to also identify the
> SCIM version? This way the relevant listeners will check for the correct
> version and only execute.
>
> For management console operations we won't have a threadlocal set. In that
> case one of the listeners (ideally SCIM2) will execute and do the necessary
> changes to the user store, so that thereafter even if we do calls using
> SCIM 1.1. it should work without a problem.
>
> This way we are not coupling the listeners, each version works
> independently, and management console will depend on latest version. Even
> if tomorrow SCIM 3.0 comes out we don't need to change existing code :)
>
> Anyone see any issue with this approach? Am I missing anything?
>
> Regards,
> Johann.
>
> On Mon, Sep 18, 2017 at 11:04 AM, Pulasthi Mahawithana  > wrote:
>
>> Hi Gayan,
>>
>> On Mon, Sep 18, 2017 at 10:43 AM, Gayan Gunawardana 
>> wrote:
>>
>>>
>>>
>>> On Mon, Sep 18, 2017 at 10:19 AM, Prabath Siriwardena 
>>> wrote:
>>>


 On Sun, Sep 17, 2017 at 9:42 PM, Gayan Gunawardana 
 wrote:

>
>
> On Mon, Sep 18, 2017 at 8:29 AM, Darshana Gunawardana <
> darsh...@wso2.com> wrote:
>
>> Hi Prabath,
>>
>> We do have two endpoints for SCIM 1.1 & SCIM 2. But there are some
>> listeners we need to enable in order to populated relevant metadata such 
>> as
>> ID, Created\Modified dates against the correct URI specified by the each
>> specification.
>>
>> Given that, different clients may use different protocol of they
>> already support, we should have ability to have both protocols working
>> simultaneously.
>>
> According to this even single client can use SCIM 1.1 and 2.0
> simultaneously right ?
> Yes so ideally it should work but in practice it will bring some
> complexities. We have to test use-cases like creating user with SCIM 1.1
> and updating with SCIM 2.0.
> IMO we should support only one protocol at a time but not both.
>

 Can we please identify those complexities...?

 We cannot terminate support for SCIM 1.1 - both have to co-exist for
 few releases till we deprecate SCIM 1.1. If both cannot co-exist then we
 need to review our design...

>>> What I am suggesting is not to terminate support for SCIM 1.1 but at a
>>> time enable only.
>>> Enable SCIM 1.1 as default option. If client applications work with SCIM
>>> 1.1 they can continue. If client applications want to work with SCIM 2.0 we
>>> should be able to enable SCIM 2.0 by disabling SCIM 1.1 from
>>> configurations.
>>>
>>
>> What I initially suggested was the other way around. That is having SCIM
>> 2 as default and make SCIM 1.1 to enable with a config change. The reason
>> is, for any new users, they can use the newer standard OOTB without any
>> config changes. And for existing users who are using SCIM 1.1 clients will
>> anyway need to migrate to 5.4.0. During the migration, we keep the existing
>> configuration. So for them SCIM 1.1 will be enabled by default after the
>> migration process (They may switch later if needed).
>>
>> Anyway, as Darshana mentioned, We'll look for the possibility of having
>> both of them enabled at once.
>>
>>>
 Thanks & regards,
 -Prabath



>
>> Created: https://wso2.org/jira/browse/IDENTITY-6458 to track this.
>>
>> Thanks,
>>
>> On Mon, Sep 18, 2017 at 8:11 AM, Darshana Gunawardana <
>> darsh...@wso2.com> wrote:
>>
>>> Hi Gayan,
>>>
>>> Due to the limitation we have in the user core level, we don't
>>> support complex filtering, pagination, sorting. Refer [1] & [2].
>>>
>>> [1] https://github.com/wso2-extensions/identity-inbound-prov
>>> isioning-scim2/blob/v1.0.5/components/org.wso2.carbon.identi
>>> ty.scim2.common/src/main/java/org/wso2/carbon/identity/scim2
>>> /common/impl/SCIMUserManager.java#L834
>>> [2] https://github.com/wso2-extensions/identity-inbound-prov
>>> isioning-scim2/blob/v1.0.5/components/org.wso2.carbon.identi
>>> ty.scim2.common/src/main/java/org/wso2/carbon/identity/scim2
>>> /common/impl/SCIMUserManager.java#L793
>>>
>>> Thanks,
>>>
>>> On Mon, Sep 18, 2017 at 7:50 AM, Gayan Gunawardana 
>>> wrote:
>>>


 On Mon, Sep 11, 2017 at 9:14 PM, Vindula Jayawardana <
 vindula...@cse.mrt.ac.lk> wrote:

> Hi,
>
> +1 for this as the global adoption of SCIM 2.0 is right on the
> track as Ishara has