Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Mohanadarshan Vivekanandalingam]

On Mon, Aug 15, 2016 at 3:07 AM, Johann Nallathamby  wrote:

>
>
>>
>> But IMO, this is a notification task isn't. Then, I don't see a need to
>> worry much about performance and throughput..
>>
>
> Performance is not the issue here.
>
>
>> And I don't think it is correct to change the generic notification event
>> publisher component as something which does some IS specific operation
>> which violates its design expectation and looks like a hack ..
>>
>
> That is not quite correct Mohan. Adding claim values to email template is
> nothing IS specific. The CEP's output adaptor framework is used by many
> WSO2 products for notifications. This is a common and valid requirement for
> all the products IMHO. It is with that intension we implemented HTML
> templating support and multi language template support in CEP itself.
> Similarly this is also another requirement.
>
> And I also don't think it is violating the existing architecture.
>
> I would think the same if the adaptors are implemented like a library
> which don't have any other carbon runtime dependency. However fetching the
> email template from registry is also done within the adaptor. So there is
> no good explanation to say fetching user claims from user store violates
> the adaptor design.
>
> If there was a layer before the adaptor layer which handles fetching email
> templates from registry, then in the same layer we can enrich the event
> stream with user claims defined in that template. So by the time the event
> reaches the adaptor all necessary values are in the stream.
>
> What I am saying is whether this code is in the adaptor or in a higher
> layer, ultimately it must be in CEP for all the products to use it. That is
> why we reckon that it is an improvement for CEP.
>

Thanks Johann for the explanation.. I like to add some comments on above..
Event Publisher component consists of two layers, they are event enrichment
layer and transport layer.. Calling registry to get the template, multi
language support and HTML things are acceptable in event enrichment layer
since they used to enhance the event but retrieving claims to create/fill
the event cannot be considered as event enrichment IMHO..

Eventhough IS only sends event specific properties, it sends those
properties through stream then they are considered as event in the eventing
context.. In Event publisher layer, we can do enrichment for that event..

Anyway, Suho and myself had a quick offline chat last Friday.. After that,
I also have a mixed opinion like Suho have.. Anyway, we thought best
approach would be providing an extension point for event enrichment layer..
Then whenever need to perform any operation like above we can use that
extension.. Hope that solves above requirement..

Regards,
Mohan


>
> Regards,
> Johann.
>
>
>> Let's have a quick chat if that help..
>>
>> Thanks,
>> Mohan
>>
>>
>>
>>>
>>> Regards
>>> Suho
>>>
>>> On Thu, Aug 11, 2016 at 4:42 PM, Mohanadarshan Vivekanandalingam <
>>> mo...@wso2.com> wrote:
>>>


 On Thu, Aug 11, 2016 at 11:32 AM, Indunil Upeksha Rathnayake <
 indu...@wso2.com> wrote:

> Hi Suhothayan,
>
> You can refer [1] for the current approach we have taken in IS side
> when improving notification sending with siddhi streams. As per the
> discussion we had previously, this approach has been taken in order to
> avoid the performance degradation due to the redundant loading of email
> template in IS and analytics. The main reason for the redundant loading is
> that only in IS side, the user claims can be loaded which needs for 
> filling
> out the placeholders in email template.
>
> As per the current implementation you are having, we can provide the
> registry path and let the email template get loaded in analytics side. For
> that there has to be some improvement in analytics side to get the user
> claims from user store and filling out the template with those claim
> values. So that without loading the email template from IS side, we can do
> it in analytics side.
>
> So the suggested improvements as follows.
> *IS side:*
>
>
>
>
>
>
> *1) Modified the publisher definition to include registry path of the
> email template, specifying the notification type and locale as
> placeholders2) When an email notification need to be send, an arbitrary 
> map
> (including the data needs to load the email template from registry) will 
> be
> published to the streamAnalytics side:1) Load the email template from the
> registry (use the arbitrary data values we have provided)2) Extract the
> placeholders in email template3) Get the user claims from user store and
> fill out the placeholders in the template with the necessary claim values*
>

 Above [1] and [2] are already implemented in Event Publisher level and
 can be usable for above usecase.. But [2] (which is 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Johann Nallathamby]

Hi Mohan,

On Fri, Aug 12, 2016 at 1:08 AM, Mohanadarshan Vivekanandalingam <
mo...@wso2.com> wrote:

>
>
> On Thu, Aug 11, 2016 at 11:45 PM, Sriskandarajah Suhothayan  > wrote:
>
>> I think getting the claims will improve the message formatting when
>> sending the message, and based on the discussion with Johann they cannot
>> determine what claims the message formatting will need. if IS need to send
>> the claims it has to also read the message to understand the necessary
>> claims.
>>
>> Any suggestions ?
>>
>
> Hmm.. Message is created/originated from IS side, Isn't it ? Then, why
> can't we get whatever claims that required and send them..
>

Message is not created in IS side. From IS side we only send the event
specific properties. The admin can define whatever claims he wants in the
email template. These are two independent actions.


> Sorry, I don't see any overall impact because of that (Only thing is there
> going to be another registry access to get the template and load relevant
> claims). If this is an huge overhead then can't we send all the claims with
> in the event (Not sure whether it is acceptable)..
>

We can't send all the claims because there can be sensitive information.
Its not a scalable solution.


>
> But IMO, this is a notification task isn't. Then, I don't see a need to
> worry much about performance and throughput..
>

Performance is not the issue here.


> And I don't think it is correct to change the generic notification event
> publisher component as something which does some IS specific operation
> which violates its design expectation and looks like a hack ..
>

That is not quite correct Mohan. Adding claim values to email template is
nothing IS specific. The CEP's output adaptor framework is used by many
WSO2 products for notifications. This is a common and valid requirement for
all the products IMHO. It is with that intension we implemented HTML
templating support and multi language template support in CEP itself.
Similarly this is also another requirement.

And I also don't think it is violating the existing architecture.

I would think the same if the adaptors are implemented like a library which
don't have any other carbon runtime dependency. However fetching the email
template from registry is also done within the adaptor. So there is no good
explanation to say fetching user claims from user store violates the
adaptor design.

If there was a layer before the adaptor layer which handles fetching email
templates from registry, then in the same layer we can enrich the event
stream with user claims defined in that template. So by the time the event
reaches the adaptor all necessary values are in the stream.

What I am saying is whether this code is in the adaptor or in a higher
layer, ultimately it must be in CEP for all the products to use it. That is
why we reckon that it is an improvement for CEP.

Regards,
Johann.


> Let's have a quick chat if that help..
>
> Thanks,
> Mohan
>
>
>
>>
>> Regards
>> Suho
>>
>> On Thu, Aug 11, 2016 at 4:42 PM, Mohanadarshan Vivekanandalingam <
>> mo...@wso2.com> wrote:
>>
>>>
>>>
>>> On Thu, Aug 11, 2016 at 11:32 AM, Indunil Upeksha Rathnayake <
>>> indu...@wso2.com> wrote:
>>>
 Hi Suhothayan,

 You can refer [1] for the current approach we have taken in IS side
 when improving notification sending with siddhi streams. As per the
 discussion we had previously, this approach has been taken in order to
 avoid the performance degradation due to the redundant loading of email
 template in IS and analytics. The main reason for the redundant loading is
 that only in IS side, the user claims can be loaded which needs for filling
 out the placeholders in email template.

 As per the current implementation you are having, we can provide the
 registry path and let the email template get loaded in analytics side. For
 that there has to be some improvement in analytics side to get the user
 claims from user store and filling out the template with those claim
 values. So that without loading the email template from IS side, we can do
 it in analytics side.

 So the suggested improvements as follows.
 *IS side:*






 *1) Modified the publisher definition to include registry path of the
 email template, specifying the notification type and locale as
 placeholders2) When an email notification need to be send, an arbitrary map
 (including the data needs to load the email template from registry) will be
 published to the streamAnalytics side:1) Load the email template from the
 registry (use the arbitrary data values we have provided)2) Extract the
 placeholders in email template3) Get the user claims from user store and
 fill out the placeholders in the template with the necessary claim values*

>>>
>>> Above [1] and [2] are already implemented in Event Publisher level and
>>> can be usable for 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Mohanadarshan Vivekanandalingam]

On Thu, Aug 11, 2016 at 11:45 PM, Sriskandarajah Suhothayan 
wrote:

> I think getting the claims will improve the message formatting when
> sending the message, and based on the discussion with Johann they cannot
> determine what claims the message formatting will need. if IS need to send
> the claims it has to also read the message to understand the necessary
> claims.
>
> Any suggestions ?
>

Hmm.. Message is created/originated from IS side, Isn't it ? Then, why
can't we get whatever claims that required and send them.. Sorry, I don't
see any overall impact because of that (Only thing is there going to be
another registry access to get the template and load relevant claims). If
this is an huge overhead then can't we send all the claims with in the
event (Not sure whether it is acceptable)..

But IMO, this is a notification task isn't. Then, I don't see a need to
worry much about performance and throughput.. And I don't think it is
correct to change the generic notification event publisher component as
something which does some IS specific operation which violates its design
expectation and looks like a hack ..

Let's have a quick chat if that help..

Thanks,
Mohan



>
> Regards
> Suho
>
> On Thu, Aug 11, 2016 at 4:42 PM, Mohanadarshan Vivekanandalingam <
> mo...@wso2.com> wrote:
>
>>
>>
>> On Thu, Aug 11, 2016 at 11:32 AM, Indunil Upeksha Rathnayake <
>> indu...@wso2.com> wrote:
>>
>>> Hi Suhothayan,
>>>
>>> You can refer [1] for the current approach we have taken in IS side when
>>> improving notification sending with siddhi streams. As per the discussion
>>> we had previously, this approach has been taken in order to avoid the
>>> performance degradation due to the redundant loading of email template in
>>> IS and analytics. The main reason for the redundant loading is that only in
>>> IS side, the user claims can be loaded which needs for filling out the
>>> placeholders in email template.
>>>
>>> As per the current implementation you are having, we can provide the
>>> registry path and let the email template get loaded in analytics side. For
>>> that there has to be some improvement in analytics side to get the user
>>> claims from user store and filling out the template with those claim
>>> values. So that without loading the email template from IS side, we can do
>>> it in analytics side.
>>>
>>> So the suggested improvements as follows.
>>> *IS side:*
>>>
>>>
>>>
>>>
>>>
>>>
>>> *1) Modified the publisher definition to include registry path of the
>>> email template, specifying the notification type and locale as
>>> placeholders2) When an email notification need to be send, an arbitrary map
>>> (including the data needs to load the email template from registry) will be
>>> published to the streamAnalytics side:1) Load the email template from the
>>> registry (use the arbitrary data values we have provided)2) Extract the
>>> placeholders in email template3) Get the user claims from user store and
>>> fill out the placeholders in the template with the necessary claim values*
>>>
>>
>> Above [1] and [2] are already implemented in Event Publisher level and
>> can be usable for above usecase.. But [2] (which is mentioned as an
>> improvement for Analytics side) is not valid IMO.. That is not something
>> that we need to handle in analytics or Event Publisher side, it is
>> architecturally incorrect to handle in that level.. What need to be done
>> is, we need to get those claims from identity level and send those relevant
>> claims in the event itself..
>>
>> What is the reason for defining [3] as an improvement for analytics ?
>>
>> Thanks,
>> Mohan
>>
>>
>>>
>>>
>>> We have used two prefixes in placeholders of email templates as
>>> "user.claim.identity" and "user.claim", in order to specify that the
>>> placeholders has to be filled with an identity claim and other wso2 claim
>>> respectively. The claim URIs which we are using when retrieving necessary
>>> user claims for the email templates, will be generated appending necessary
>>> prefix to the "http://wso2.org/claims/;. As an example if the
>>> placeholder is "user.claim.givenname", the claim URI should be "
>>> http://wso2.org/claims/givenname;. So that placeholder has to be filled
>>> with the user claim value corresponding to the above mentioned claim URI.
>>> You can refer [2] for the implementation done in IS side, we can move that
>>> logic to analytics side.
>>>
>>> [1] https://github.com/wso2-extensions/identity-event-handler-no
>>> tification/pull/26/files
>>> [2] https://github.com/wso2-extensions/identity-event-handler-no
>>> tification/pull/26/files#diff-2200b351eeef81ebbb5ea7f0d1f1ecb7R119
>>>
>>> Thanks and Regards
>>>
>>> On Tue, Aug 9, 2016 at 9:50 PM, Sriskandarajah Suhothayan >> > wrote:
>>>
 Based on the chat with Johann he suggested to support claims at event
 publisher.
 @Indunil, can you get the full requirements and update the thread.

 Regards
 Suho

 On 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Sriskandarajah Suhothayan]

I think getting the claims will improve the message formatting when sending
the message, and based on the discussion with Johann they cannot determine
what claims the message formatting will need. if IS need to send the claims
it has to also read the message to understand the necessary claims.

Any suggestions ?

Regards
Suho

On Thu, Aug 11, 2016 at 4:42 PM, Mohanadarshan Vivekanandalingam <
mo...@wso2.com> wrote:

>
>
> On Thu, Aug 11, 2016 at 11:32 AM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi Suhothayan,
>>
>> You can refer [1] for the current approach we have taken in IS side when
>> improving notification sending with siddhi streams. As per the discussion
>> we had previously, this approach has been taken in order to avoid the
>> performance degradation due to the redundant loading of email template in
>> IS and analytics. The main reason for the redundant loading is that only in
>> IS side, the user claims can be loaded which needs for filling out the
>> placeholders in email template.
>>
>> As per the current implementation you are having, we can provide the
>> registry path and let the email template get loaded in analytics side. For
>> that there has to be some improvement in analytics side to get the user
>> claims from user store and filling out the template with those claim
>> values. So that without loading the email template from IS side, we can do
>> it in analytics side.
>>
>> So the suggested improvements as follows.
>> *IS side:*
>>
>>
>>
>>
>>
>>
>> *1) Modified the publisher definition to include registry path of the
>> email template, specifying the notification type and locale as
>> placeholders2) When an email notification need to be send, an arbitrary map
>> (including the data needs to load the email template from registry) will be
>> published to the streamAnalytics side:1) Load the email template from the
>> registry (use the arbitrary data values we have provided)2) Extract the
>> placeholders in email template3) Get the user claims from user store and
>> fill out the placeholders in the template with the necessary claim values*
>>
>
> Above [1] and [2] are already implemented in Event Publisher level and can
> be usable for above usecase.. But [2] (which is mentioned as an improvement
> for Analytics side) is not valid IMO.. That is not something that we need
> to handle in analytics or Event Publisher side, it is architecturally
> incorrect to handle in that level.. What need to be done is, we need to get
> those claims from identity level and send those relevant claims in the
> event itself..
>
> What is the reason for defining [3] as an improvement for analytics ?
>
> Thanks,
> Mohan
>
>
>>
>>
>> We have used two prefixes in placeholders of email templates as
>> "user.claim.identity" and "user.claim", in order to specify that the
>> placeholders has to be filled with an identity claim and other wso2 claim
>> respectively. The claim URIs which we are using when retrieving necessary
>> user claims for the email templates, will be generated appending necessary
>> prefix to the "http://wso2.org/claims/;. As an example if the
>> placeholder is "user.claim.givenname", the claim URI should be "
>> http://wso2.org/claims/givenname;. So that placeholder has to be filled
>> with the user claim value corresponding to the above mentioned claim URI.
>> You can refer [2] for the implementation done in IS side, we can move that
>> logic to analytics side.
>>
>> [1] https://github.com/wso2-extensions/identity-event-handler-
>> notification/pull/26/files
>> [2] https://github.com/wso2-extensions/identity-event-handler-
>> notification/pull/26/files#diff-2200b351eeef81ebbb5ea7f0d1f1ecb7R119
>>
>> Thanks and Regards
>>
>> On Tue, Aug 9, 2016 at 9:50 PM, Sriskandarajah Suhothayan 
>> wrote:
>>
>>> Based on the chat with Johann he suggested to support claims at event
>>> publisher.
>>> @Indunil, can you get the full requirements and update the thread.
>>>
>>> Regards
>>> Suho
>>>
>>> On Mon, Aug 1, 2016 at 11:24 PM, Mohanadarshan Vivekanandalingam <
>>> mo...@wso2.com> wrote:
>>>


 On Mon, Aug 1, 2016 at 8:38 PM, Indunil Upeksha Rathnayake <
 indu...@wso2.com> wrote:

> Hi Suhothayan,
>
> Hi Indunil,

 I like to add some comments on this.. Please find them below..


> There was an issue in EventPublisherServiceDS where
> setConfigurationContextService() method get invoked after the bundle
> get activated. Due to that, when we are trying to invoke
> deployEventPublisherConfiguration() of EventPublisherService from the
> activate method of an osgi bundle in IS side, it's receiving a null
> pointer(Since it refers the ConfigurationContextService object in
> EventPublisherServiceValueHolder). I think you can resolve it by
> changing the osgi reference cardinality in [1] as "1..1"(Mandatory), if
> there is no specific reason for making it optional.
>

 There is a 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Mohanadarshan Vivekanandalingam]

On Thu, Aug 11, 2016 at 11:32 AM, Indunil Upeksha Rathnayake <
indu...@wso2.com> wrote:

> Hi Suhothayan,
>
> You can refer [1] for the current approach we have taken in IS side when
> improving notification sending with siddhi streams. As per the discussion
> we had previously, this approach has been taken in order to avoid the
> performance degradation due to the redundant loading of email template in
> IS and analytics. The main reason for the redundant loading is that only in
> IS side, the user claims can be loaded which needs for filling out the
> placeholders in email template.
>
> As per the current implementation you are having, we can provide the
> registry path and let the email template get loaded in analytics side. For
> that there has to be some improvement in analytics side to get the user
> claims from user store and filling out the template with those claim
> values. So that without loading the email template from IS side, we can do
> it in analytics side.
>
> So the suggested improvements as follows.
> *IS side:*
>
>
>
>
>
>
> *1) Modified the publisher definition to include registry path of the
> email template, specifying the notification type and locale as
> placeholders2) When an email notification need to be send, an arbitrary map
> (including the data needs to load the email template from registry) will be
> published to the streamAnalytics side:1) Load the email template from the
> registry (use the arbitrary data values we have provided)2) Extract the
> placeholders in email template3) Get the user claims from user store and
> fill out the placeholders in the template with the necessary claim values*
>

Above [1] and [2] are already implemented in Event Publisher level and can
be usable for above usecase.. But [2] (which is mentioned as an improvement
for Analytics side) is not valid IMO.. That is not something that we need
to handle in analytics or Event Publisher side, it is architecturally
incorrect to handle in that level.. What need to be done is, we need to get
those claims from identity level and send those relevant claims in the
event itself..

What is the reason for defining [3] as an improvement for analytics ?

Thanks,
Mohan


>
>
> We have used two prefixes in placeholders of email templates as
> "user.claim.identity" and "user.claim", in order to specify that the
> placeholders has to be filled with an identity claim and other wso2 claim
> respectively. The claim URIs which we are using when retrieving necessary
> user claims for the email templates, will be generated appending necessary
> prefix to the "http://wso2.org/claims/;. As an example if the placeholder
> is "user.claim.givenname", the claim URI should be "
> http://wso2.org/claims/givenname;. So that placeholder has to be filled
> with the user claim value corresponding to the above mentioned claim URI.
> You can refer [2] for the implementation done in IS side, we can move that
> logic to analytics side.
>
> [1] https://github.com/wso2-extensions/identity-event-
> handler-notification/pull/26/files
> [2] https://github.com/wso2-extensions/identity-event-
> handler-notification/pull/26/files#diff-2200b351eeef81ebbb5ea7f0d1f1ec
> b7R119
>
> Thanks and Regards
>
> On Tue, Aug 9, 2016 at 9:50 PM, Sriskandarajah Suhothayan 
> wrote:
>
>> Based on the chat with Johann he suggested to support claims at event
>> publisher.
>> @Indunil, can you get the full requirements and update the thread.
>>
>> Regards
>> Suho
>>
>> On Mon, Aug 1, 2016 at 11:24 PM, Mohanadarshan Vivekanandalingam <
>> mo...@wso2.com> wrote:
>>
>>>
>>>
>>> On Mon, Aug 1, 2016 at 8:38 PM, Indunil Upeksha Rathnayake <
>>> indu...@wso2.com> wrote:
>>>
 Hi Suhothayan,

 Hi Indunil,
>>>
>>> I like to add some comments on this.. Please find them below..
>>>
>>>
 There was an issue in EventPublisherServiceDS where
 setConfigurationContextService() method get invoked after the bundle
 get activated. Due to that, when we are trying to invoke
 deployEventPublisherConfiguration() of EventPublisherService from the
 activate method of an osgi bundle in IS side, it's receiving a null
 pointer(Since it refers the ConfigurationContextService object in
 EventPublisherServiceValueHolder). I think you can resolve it by
 changing the osgi reference cardinality in [1] as "1..1"(Mandatory), if
 there is no specific reason for making it optional.

>>>
>>> There is a valid reason for this..
>>> I believe, as you know we cannot guarantee about OSGI bundle loading in
>>> carbon environment.. In this case, there is a possibility where axis2
>>> deployment can start before bundle activation of a OSGI component. To avoid
>>> this we'll follow a similar approach like below,
>>>
>>> 
>>>
>>>org.wso2.carbon.event.publisher.core.EventPublisherService
>>>
>>> 
>>>
>>> Here, we are adding the reference of the corresponding OSGI service
>>> which is exposed by relevant OSGI module.. If you 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Indunil Upeksha Rathnayake]

Hi Suhothayan,

You can refer [1] for the current approach we have taken in IS side when
improving notification sending with siddhi streams. As per the discussion
we had previously, this approach has been taken in order to avoid the
performance degradation due to the redundant loading of email template in
IS and analytics. The main reason for the redundant loading is that only in
IS side, the user claims can be loaded which needs for filling out the
placeholders in email template.

As per the current implementation you are having, we can provide the
registry path and let the email template get loaded in analytics side. For
that there has to be some improvement in analytics side to get the user
claims from user store and filling out the template with those claim
values. So that without loading the email template from IS side, we can do
it in analytics side.

So the suggested improvements as follows.
*IS side:*






*1) Modified the publisher definition to include registry path of the email
template, specifying the notification type and locale as placeholders2)
When an email notification need to be send, an arbitrary map (including the
data needs to load the email template from registry) will be published to
the streamAnalytics side:1) Load the email template from the registry (use
the arbitrary data values we have provided)2) Extract the placeholders in
email template3) Get the user claims from user store and fill out the
placeholders in the template with the necessary claim values*

We have used two prefixes in placeholders of email templates as
"user.claim.identity" and "user.claim", in order to specify that the
placeholders has to be filled with an identity claim and other wso2 claim
respectively. The claim URIs which we are using when retrieving necessary
user claims for the email templates, will be generated appending necessary
prefix to the "http://wso2.org/claims/;. As an example if the placeholder
is "user.claim.givenname", the claim URI should be "http://wso2.org/claims/
givenname". So that placeholder has to be filled with the user claim value
corresponding to the above mentioned claim URI. You can refer [2] for the
implementation done in IS side, we can move that logic to analytics side.

[1]
https://github.com/wso2-extensions/identity-event-handler-notification/pull/26/files
[2]
https://github.com/wso2-extensions/identity-event-handler-notification/pull/26/files#diff-2200b351eeef81ebbb5ea7f0d1f1ecb7R119

Thanks and Regards

On Tue, Aug 9, 2016 at 9:50 PM, Sriskandarajah Suhothayan 
wrote:

> Based on the chat with Johann he suggested to support claims at event
> publisher.
> @Indunil, can you get the full requirements and update the thread.
>
> Regards
> Suho
>
> On Mon, Aug 1, 2016 at 11:24 PM, Mohanadarshan Vivekanandalingam <
> mo...@wso2.com> wrote:
>
>>
>>
>> On Mon, Aug 1, 2016 at 8:38 PM, Indunil Upeksha Rathnayake <
>> indu...@wso2.com> wrote:
>>
>>> Hi Suhothayan,
>>>
>>> Hi Indunil,
>>
>> I like to add some comments on this.. Please find them below..
>>
>>
>>> There was an issue in EventPublisherServiceDS where
>>> setConfigurationContextService() method get invoked after the bundle
>>> get activated. Due to that, when we are trying to invoke
>>> deployEventPublisherConfiguration() of EventPublisherService from the
>>> activate method of an osgi bundle in IS side, it's receiving a null
>>> pointer(Since it refers the ConfigurationContextService object in
>>> EventPublisherServiceValueHolder). I think you can resolve it by
>>> changing the osgi reference cardinality in [1] as "1..1"(Mandatory), if
>>> there is no specific reason for making it optional.
>>>
>>
>> There is a valid reason for this..
>> I believe, as you know we cannot guarantee about OSGI bundle loading in
>> carbon environment.. In this case, there is a possibility where axis2
>> deployment can start before bundle activation of a OSGI component. To avoid
>> this we'll follow a similar approach like below,
>>
>> 
>>
>>org.wso2.carbon.event.publisher.core.EventPublisherService
>>
>> 
>>
>> Here, we are adding the reference of the corresponding OSGI service which
>> is exposed by relevant OSGI module.. If you want to use above approach
>> (Axis2RequiredServices), we cannot have 1..1 mapping for
>> ConfigurationContextService since it causes cyclic dependency and affects
>> bundle loading..
>>
>> In IS side we were able to get rid of the null pointer by adding an osgi
>>> reference for ConfigurationContextService in the service component and
>>> invoked the deployEventPublisherConfiguration() in activate() method.
>>>
>>
>> No, above solution is not correct and will not work all the time.. There
>> is a possibility where you'll encounter same issue when
>> ConfigurationContextService is bind to you component first and takes
>> sometime to resolve for Event Publisher..
>>
>> What is the usecase for creating an Event Publisher in server restart ?
>> Can you ship the pack with an Event 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Sriskandarajah Suhothayan]

Based on the chat with Johann he suggested to support claims at event
publisher.
@Indunil, can you get the full requirements and update the thread.

Regards
Suho

On Mon, Aug 1, 2016 at 11:24 PM, Mohanadarshan Vivekanandalingam <
mo...@wso2.com> wrote:

>
>
> On Mon, Aug 1, 2016 at 8:38 PM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi Suhothayan,
>>
>> Hi Indunil,
>
> I like to add some comments on this.. Please find them below..
>
>
>> There was an issue in EventPublisherServiceDS where
>> setConfigurationContextService() method get invoked after the bundle get
>> activated. Due to that, when we are trying to invoke
>> deployEventPublisherConfiguration() of EventPublisherService from the
>> activate method of an osgi bundle in IS side, it's receiving a null
>> pointer(Since it refers the ConfigurationContextService object in
>> EventPublisherServiceValueHolder). I think you can resolve it by
>> changing the osgi reference cardinality in [1] as "1..1"(Mandatory), if
>> there is no specific reason for making it optional.
>>
>
> There is a valid reason for this..
> I believe, as you know we cannot guarantee about OSGI bundle loading in
> carbon environment.. In this case, there is a possibility where axis2
> deployment can start before bundle activation of a OSGI component. To avoid
> this we'll follow a similar approach like below,
>
> 
>
>org.wso2.carbon.event.publisher.core.EventPublisherService
>
> 
>
> Here, we are adding the reference of the corresponding OSGI service which
> is exposed by relevant OSGI module.. If you want to use above approach
> (Axis2RequiredServices), we cannot have 1..1 mapping for
> ConfigurationContextService since it causes cyclic dependency and affects
> bundle loading..
>
> In IS side we were able to get rid of the null pointer by adding an osgi
>> reference for ConfigurationContextService in the service component and
>> invoked the deployEventPublisherConfiguration() in activate() method.
>>
>
> No, above solution is not correct and will not work all the time.. There
> is a possibility where you'll encounter same issue when
> ConfigurationContextService is bind to you component first and takes
> sometime to resolve for Event Publisher..
>
> What is the usecase for creating an Event Publisher in server restart ?
> Can you ship the pack with an Event Publisher or deploy an event publisher
> for first event if it is not there..
>
>
>> And also there was an issue in filling out dynamic properties of an
>> output adapter from the arbitrary data values, and sent a PR for that.
>> Please review and merge the PR in [2].
>>
>
> Thanks, Merged it..
>
> Regards,
> Mohan
>
>
>>
>> [1] https://github.com/wso2/carbon-analytics-common/blob/
>> master/components/event-publisher/org.wso2.carbon.
>> event.publisher.core/src/main/java/org/wso2/carbon/event/
>> publisher/core/internal/ds/EventPublisherServiceDS.java#L56
>> [2] https://github.com/wso2/carbon-analytics-common/pull/306/files
>>
>> Thanks and Regards
>>
>> On Mon, Aug 1, 2016 at 3:06 PM, Sriskandarajah Suhothayan 
>> wrote:
>>
>>> HI Indunil
>>>
>>> Any update on this? Was the provided solution working?
>>>
>>> We released CEP 4.2-RC1. If we need new features/improvements for this
>>> effort, we can incorporate them in the next component release.
>>>
>>> Regards
>>> Suho
>>>
>>> On Fri, Jul 22, 2016 at 3:10 PM, Sriskandarajah Suhothayan <
>>> s...@wso2.com> wrote:
>>>


 On Fri, Jul 22, 2016 at 3:00 PM, Johann Nallathamby 
 wrote:

>
>
> On Fri, Jul 22, 2016 at 8:33 AM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi,
>>
>> On Fri, Jul 22, 2016 at 12:28 PM, Sriskandarajah Suhothayan <
>> s...@wso2.com> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
>>> indu...@wso2.com> wrote:
>>>
 Hi,

 Please find the meeting notes in [1].  I have following
 considerations regarding the improvements we have discussed.

 (1) Even though we have configured to load the email template from
 EventPublisher(analytics side), the placeholder values has to be sent 
 as
 meta data/correlation data/payload data/arbitrary data, since in 
 analytics
 side, the user claim values are not getting from the user store.
 In order to send the placeholder values from IS side, anyway we
 have to load the email template and retrieve the placeholders. So as I 
 have
 understood, for email notifications, it's not needed to use the email
 template loading part in analytics, since it'll be a redundant task. 
 (Refer
 [2])

>>>
>>> Here we can set the claim values as arbitrary data, and the
>>> notification specific details as the meta, correlation & payload data.
>>> Then we can use the template 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Mohanadarshan Vivekanandalingam]

On Mon, Aug 1, 2016 at 8:38 PM, Indunil Upeksha Rathnayake  wrote:

> Hi Suhothayan,
>
> Hi Indunil,

I like to add some comments on this.. Please find them below..


> There was an issue in EventPublisherServiceDS where
> setConfigurationContextService() method get invoked after the bundle get
> activated. Due to that, when we are trying to invoke
> deployEventPublisherConfiguration() of EventPublisherService from the
> activate method of an osgi bundle in IS side, it's receiving a null
> pointer(Since it refers the ConfigurationContextService object in
> EventPublisherServiceValueHolder). I think you can resolve it by changing
> the osgi reference cardinality in [1] as "1..1"(Mandatory), if there is no
> specific reason for making it optional.
>

There is a valid reason for this..
I believe, as you know we cannot guarantee about OSGI bundle loading in
carbon environment.. In this case, there is a possibility where axis2
deployment can start before bundle activation of a OSGI component. To avoid
this we'll follow a similar approach like below,



   org.wso2.carbon.event.publisher.core.EventPublisherService



Here, we are adding the reference of the corresponding OSGI service which
is exposed by relevant OSGI module.. If you want to use above approach
(Axis2RequiredServices), we cannot have 1..1 mapping for
ConfigurationContextService since it causes cyclic dependency and affects
bundle loading..

In IS side we were able to get rid of the null pointer by adding an osgi
> reference for ConfigurationContextService in the service component and
> invoked the deployEventPublisherConfiguration() in activate() method.
>

No, above solution is not correct and will not work all the time.. There is
a possibility where you'll encounter same issue when
ConfigurationContextService is bind to you component first and takes
sometime to resolve for Event Publisher..

What is the usecase for creating an Event Publisher in server restart ? Can
you ship the pack with an Event Publisher or deploy an event publisher for
first event if it is not there..


> And also there was an issue in filling out dynamic properties of an output
> adapter from the arbitrary data values, and sent a PR for that. Please
> review and merge the PR in [2].
>

Thanks, Merged it..

Regards,
Mohan


>
> [1]
> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/ds/EventPublisherServiceDS.java#L56
> [2] https://github.com/wso2/carbon-analytics-common/pull/306/files
>
> Thanks and Regards
>
> On Mon, Aug 1, 2016 at 3:06 PM, Sriskandarajah Suhothayan 
> wrote:
>
>> HI Indunil
>>
>> Any update on this? Was the provided solution working?
>>
>> We released CEP 4.2-RC1. If we need new features/improvements for this
>> effort, we can incorporate them in the next component release.
>>
>> Regards
>> Suho
>>
>> On Fri, Jul 22, 2016 at 3:10 PM, Sriskandarajah Suhothayan > > wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 3:00 PM, Johann Nallathamby 
>>> wrote:
>>>


 On Fri, Jul 22, 2016 at 8:33 AM, Indunil Upeksha Rathnayake <
 indu...@wso2.com> wrote:

> Hi,
>
> On Fri, Jul 22, 2016 at 12:28 PM, Sriskandarajah Suhothayan <
> s...@wso2.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
>> indu...@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> Please find the meeting notes in [1].  I have following
>>> considerations regarding the improvements we have discussed.
>>>
>>> (1) Even though we have configured to load the email template from
>>> EventPublisher(analytics side), the placeholder values has to be sent as
>>> meta data/correlation data/payload data/arbitrary data, since in 
>>> analytics
>>> side, the user claim values are not getting from the user store.
>>> In order to send the placeholder values from IS side, anyway we have
>>> to load the email template and retrieve the placeholders. So as I have
>>> understood, for email notifications, it's not needed to use the email
>>> template loading part in analytics, since it'll be a redundant task. 
>>> (Refer
>>> [2])
>>>
>>
>> Here we can set the claim values as arbitrary data, and the
>> notification specific details as the meta, correlation & payload data.
>> Then we can use the template loading only at the analytics side.
>>
> In this case, from IS side, without parsing only the user claims
> needed for a particular email template(i.e.user claim values for the
> placeholders in email template), we have to pass all the user claims as
> arbitrary data values. In that case there's no need for loading the
> template from the registry in IS side. So that in analytics side, all the
> values needed for 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Indunil Upeksha Rathnayake]

Hi Suhothayan,

There was an issue in EventPublisherServiceDS where
setConfigurationContextService() method get invoked after the bundle get
activated. Due to that, when we are trying to invoke
deployEventPublisherConfiguration() of EventPublisherService from the
activate method of an osgi bundle in IS side, it's receiving a null
pointer(Since it refers the ConfigurationContextService object in
EventPublisherServiceValueHolder). I think you can resolve it by changing
the osgi reference cardinality in [1] as "1..1"(Mandatory), if there is no
specific reason for making it optional.
In IS side we were able to get rid of the null pointer by adding an osgi
reference for ConfigurationContextService in the service component and
invoked the deployEventPublisherConfiguration() in activate() method.

And also there was an issue in filling out dynamic properties of an output
adapter from the arbitrary data values, and sent a PR for that. Please
review and merge the PR in [2].

[1]
https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/ds/EventPublisherServiceDS.java#L56
[2] https://github.com/wso2/carbon-analytics-common/pull/306/files

Thanks and Regards

On Mon, Aug 1, 2016 at 3:06 PM, Sriskandarajah Suhothayan 
wrote:

> HI Indunil
>
> Any update on this? Was the provided solution working?
>
> We released CEP 4.2-RC1. If we need new features/improvements for this
> effort, we can incorporate them in the next component release.
>
> Regards
> Suho
>
> On Fri, Jul 22, 2016 at 3:10 PM, Sriskandarajah Suhothayan 
> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 3:00 PM, Johann Nallathamby 
>> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 8:33 AM, Indunil Upeksha Rathnayake <
>>> indu...@wso2.com> wrote:
>>>
 Hi,

 On Fri, Jul 22, 2016 at 12:28 PM, Sriskandarajah Suhothayan <
 s...@wso2.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi,
>>
>> Please find the meeting notes in [1].  I have following
>> considerations regarding the improvements we have discussed.
>>
>> (1) Even though we have configured to load the email template from
>> EventPublisher(analytics side), the placeholder values has to be sent as
>> meta data/correlation data/payload data/arbitrary data, since in 
>> analytics
>> side, the user claim values are not getting from the user store.
>> In order to send the placeholder values from IS side, anyway we have
>> to load the email template and retrieve the placeholders. So as I have
>> understood, for email notifications, it's not needed to use the email
>> template loading part in analytics, since it'll be a redundant task. 
>> (Refer
>> [2])
>>
>
> Here we can set the claim values as arbitrary data, and the
> notification specific details as the meta, correlation & payload data.
> Then we can use the template loading only at the analytics side.
>
 In this case, from IS side, without parsing only the user claims needed
 for a particular email template(i.e.user claim values for the placeholders
 in email template), we have to pass all the user claims as arbitrary data
 values. In that case there's no need for loading the template from the
 registry in IS side. So that in analytics side, all the values needed for
 filling out the template will be there. Will check on that.

>>>
>>> I don't think it will be a good solution. There can be sensitive
>>> information in the claims which we can't send. So for this release it's OK
>>> if we read the template in both sides - security is more important than
>>> performance; or read it only in IS side - but additionally send all those
>>> claims as arbitrary data as well, so if some one wants can use them in CEP
>>> side by their output adaptors.
>>>
>>
>> I think then we can have a common configuration in IS side to specify
>> what are the claims that should be added to notifications.
>>
>> Regards
>> Suho
>>
>>
>>>
>>>
>
>> (2) The email templates has to be changed as follows.
>> i) if the value will be provided in an arbitrary data map, the
>> placeholder should be with a prefix "arbitrary_"
>> (ex:{{arbitrary_givenname}})
>>
> ii) if the value will be provided in an meta data map, the
>> placeholder should be changed as {{...}} (ex:{{givenname}})
>>
>> No we should not use "arbitrary_" for any cases, its internal
> information and the names should not have "arbitrary_" even if its in
> arbitrary data map or otherwise.
>
> (3) Only Text OutputMapping Content can be filled from a value in an
>> arbitrary data map using prefix "arbitrary_" .  It's not possible to use 
>> a
>> value of an arbitrary data 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Sriskandarajah Suhothayan]

HI Indunil

Any update on this? Was the provided solution working?

We released CEP 4.2-RC1. If we need new features/improvements for this
effort, we can incorporate them in the next component release.

Regards
Suho

On Fri, Jul 22, 2016 at 3:10 PM, Sriskandarajah Suhothayan 
wrote:

>
>
> On Fri, Jul 22, 2016 at 3:00 PM, Johann Nallathamby 
> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 8:33 AM, Indunil Upeksha Rathnayake <
>> indu...@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> On Fri, Jul 22, 2016 at 12:28 PM, Sriskandarajah Suhothayan <
>>> s...@wso2.com> wrote:
>>>


 On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
 indu...@wso2.com> wrote:

> Hi,
>
> Please find the meeting notes in [1].  I have following considerations
> regarding the improvements we have discussed.
>
> (1) Even though we have configured to load the email template from
> EventPublisher(analytics side), the placeholder values has to be sent as
> meta data/correlation data/payload data/arbitrary data, since in analytics
> side, the user claim values are not getting from the user store.
> In order to send the placeholder values from IS side, anyway we have
> to load the email template and retrieve the placeholders. So as I have
> understood, for email notifications, it's not needed to use the email
> template loading part in analytics, since it'll be a redundant task. 
> (Refer
> [2])
>

 Here we can set the claim values as arbitrary data, and the
 notification specific details as the meta, correlation & payload data.
 Then we can use the template loading only at the analytics side.

>>> In this case, from IS side, without parsing only the user claims needed
>>> for a particular email template(i.e.user claim values for the placeholders
>>> in email template), we have to pass all the user claims as arbitrary data
>>> values. In that case there's no need for loading the template from the
>>> registry in IS side. So that in analytics side, all the values needed for
>>> filling out the template will be there. Will check on that.
>>>
>>
>> I don't think it will be a good solution. There can be sensitive
>> information in the claims which we can't send. So for this release it's OK
>> if we read the template in both sides - security is more important than
>> performance; or read it only in IS side - but additionally send all those
>> claims as arbitrary data as well, so if some one wants can use them in CEP
>> side by their output adaptors.
>>
>
> I think then we can have a common configuration in IS side to specify what
> are the claims that should be added to notifications.
>
> Regards
> Suho
>
>
>>
>>

> (2) The email templates has to be changed as follows.
> i) if the value will be provided in an arbitrary data map, the
> placeholder should be with a prefix "arbitrary_"
> (ex:{{arbitrary_givenname}})
>
 ii) if the value will be provided in an meta data map, the
> placeholder should be changed as {{...}} (ex:{{givenname}})
>
> No we should not use "arbitrary_" for any cases, its internal
 information and the names should not have "arbitrary_" even if its in
 arbitrary data map or otherwise.

 (3) Only Text OutputMapping Content can be filled from a value in an
> arbitrary data map using prefix "arbitrary_" .  It's not possible to use a
> value of an arbitrary data map, in a Dynamic adapter properties, only a
> value from a meta data/correlation data/payload data map can be used. I
> think that need to be extended to use even an arbitrary value as a dynamic
> adapter property.(Refer [3])
>

 @Gobi can you please fix this if that's the case.


>
> (4) The default stream definitions and publisher definitions has to be
> deployed on super tenant as well as other tenants as well. And when a new
> tenant is added, those streams and publishers has to be deployed for that
> particular tenant as well.
>
> We can have a tenant creation handler to do this copying during that
 tenant creation time. WDYT?

 Really appreciate your ideas/suggestions regarding the above mentioned
> concerns.
>
> [1] Invitation: [Architecture] [Discussion] Improvement to use Siddhi
> str... @ Wed Jul 20, 2016 4:30pm - 5:30pm (IST) (indu...@wso2.com)
>
> [2]
> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L108
>
> [3]
> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/EventPublisher.java#L311
>
> Thanks and Regards
> --
> Indunil Upeksha 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Sriskandarajah Suhothayan]

On Fri, Jul 22, 2016 at 3:00 PM, Johann Nallathamby  wrote:

>
>
> On Fri, Jul 22, 2016 at 8:33 AM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi,
>>
>> On Fri, Jul 22, 2016 at 12:28 PM, Sriskandarajah Suhothayan <
>> s...@wso2.com> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
>>> indu...@wso2.com> wrote:
>>>
 Hi,

 Please find the meeting notes in [1].  I have following considerations
 regarding the improvements we have discussed.

 (1) Even though we have configured to load the email template from
 EventPublisher(analytics side), the placeholder values has to be sent as
 meta data/correlation data/payload data/arbitrary data, since in analytics
 side, the user claim values are not getting from the user store.
 In order to send the placeholder values from IS side, anyway we have to
 load the email template and retrieve the placeholders. So as I have
 understood, for email notifications, it's not needed to use the email
 template loading part in analytics, since it'll be a redundant task. (Refer
 [2])

>>>
>>> Here we can set the claim values as arbitrary data, and the notification
>>> specific details as the meta, correlation & payload data.
>>> Then we can use the template loading only at the analytics side.
>>>
>> In this case, from IS side, without parsing only the user claims needed
>> for a particular email template(i.e.user claim values for the placeholders
>> in email template), we have to pass all the user claims as arbitrary data
>> values. In that case there's no need for loading the template from the
>> registry in IS side. So that in analytics side, all the values needed for
>> filling out the template will be there. Will check on that.
>>
>
> I don't think it will be a good solution. There can be sensitive
> information in the claims which we can't send. So for this release it's OK
> if we read the template in both sides - security is more important than
> performance; or read it only in IS side - but additionally send all those
> claims as arbitrary data as well, so if some one wants can use them in CEP
> side by their output adaptors.
>

I think then we can have a common configuration in IS side to specify what
are the claims that should be added to notifications.

Regards
Suho


>
>
>>>
 (2) The email templates has to be changed as follows.
 i) if the value will be provided in an arbitrary data map, the
 placeholder should be with a prefix "arbitrary_"
 (ex:{{arbitrary_givenname}})

>>> ii) if the value will be provided in an meta data map, the
 placeholder should be changed as {{...}} (ex:{{givenname}})

 No we should not use "arbitrary_" for any cases, its internal
>>> information and the names should not have "arbitrary_" even if its in
>>> arbitrary data map or otherwise.
>>>
>>> (3) Only Text OutputMapping Content can be filled from a value in an
 arbitrary data map using prefix "arbitrary_" .  It's not possible to use a
 value of an arbitrary data map, in a Dynamic adapter properties, only a
 value from a meta data/correlation data/payload data map can be used. I
 think that need to be extended to use even an arbitrary value as a dynamic
 adapter property.(Refer [3])

>>>
>>> @Gobi can you please fix this if that's the case.
>>>
>>>

 (4) The default stream definitions and publisher definitions has to be
 deployed on super tenant as well as other tenants as well. And when a new
 tenant is added, those streams and publishers has to be deployed for that
 particular tenant as well.

 We can have a tenant creation handler to do this copying during that
>>> tenant creation time. WDYT?
>>>
>>> Really appreciate your ideas/suggestions regarding the above mentioned
 concerns.

 [1] Invitation: [Architecture] [Discussion] Improvement to use Siddhi
 str... @ Wed Jul 20, 2016 4:30pm - 5:30pm (IST) (indu...@wso2.com)

 [2]
 https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L108

 [3]
 https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/EventPublisher.java#L311

 Thanks and Regards
 --
 Indunil Upeksha Rathnayake
 Software Engineer | WSO2 Inc
 Emailindu...@wso2.com
 Mobile   0772182255



>>>
>>>
>>> --
>>>
>>> *S. Suhothayan*
>>> Associate Director / Architect & Team Lead of WSO2 Complex Event
>>> Processor
>>> *WSO2 Inc. *http://wso2.com
>>> * *
>>> lean . enterprise . middleware
>>>
>>>
>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog:
>>> 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Johann Nallathamby]

On Fri, Jul 22, 2016 at 8:33 AM, Indunil Upeksha Rathnayake <
indu...@wso2.com> wrote:

> Hi,
>
> On Fri, Jul 22, 2016 at 12:28 PM, Sriskandarajah Suhothayan  > wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
>> indu...@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> Please find the meeting notes in [1].  I have following considerations
>>> regarding the improvements we have discussed.
>>>
>>> (1) Even though we have configured to load the email template from
>>> EventPublisher(analytics side), the placeholder values has to be sent as
>>> meta data/correlation data/payload data/arbitrary data, since in analytics
>>> side, the user claim values are not getting from the user store.
>>> In order to send the placeholder values from IS side, anyway we have to
>>> load the email template and retrieve the placeholders. So as I have
>>> understood, for email notifications, it's not needed to use the email
>>> template loading part in analytics, since it'll be a redundant task. (Refer
>>> [2])
>>>
>>
>> Here we can set the claim values as arbitrary data, and the notification
>> specific details as the meta, correlation & payload data.
>> Then we can use the template loading only at the analytics side.
>>
> In this case, from IS side, without parsing only the user claims needed
> for a particular email template(i.e.user claim values for the placeholders
> in email template), we have to pass all the user claims as arbitrary data
> values. In that case there's no need for loading the template from the
> registry in IS side. So that in analytics side, all the values needed for
> filling out the template will be there. Will check on that.
>

I don't think it will be a good solution. There can be sensitive
information in the claims which we can't send. So for this release it's OK
if we read the template in both sides - security is more important than
performance; or read it only in IS side - but additionally send all those
claims as arbitrary data as well, so if some one wants can use them in CEP
side by their output adaptors.


>>
>>> (2) The email templates has to be changed as follows.
>>> i) if the value will be provided in an arbitrary data map, the
>>> placeholder should be with a prefix "arbitrary_"
>>> (ex:{{arbitrary_givenname}})
>>>
>> ii) if the value will be provided in an meta data map, the
>>> placeholder should be changed as {{...}} (ex:{{givenname}})
>>>
>>> No we should not use "arbitrary_" for any cases, its internal
>> information and the names should not have "arbitrary_" even if its in
>> arbitrary data map or otherwise.
>>
>> (3) Only Text OutputMapping Content can be filled from a value in an
>>> arbitrary data map using prefix "arbitrary_" .  It's not possible to use a
>>> value of an arbitrary data map, in a Dynamic adapter properties, only a
>>> value from a meta data/correlation data/payload data map can be used. I
>>> think that need to be extended to use even an arbitrary value as a dynamic
>>> adapter property.(Refer [3])
>>>
>>
>> @Gobi can you please fix this if that's the case.
>>
>>
>>>
>>> (4) The default stream definitions and publisher definitions has to be
>>> deployed on super tenant as well as other tenants as well. And when a new
>>> tenant is added, those streams and publishers has to be deployed for that
>>> particular tenant as well.
>>>
>>> We can have a tenant creation handler to do this copying during that
>> tenant creation time. WDYT?
>>
>> Really appreciate your ideas/suggestions regarding the above mentioned
>>> concerns.
>>>
>>> [1] Invitation: [Architecture] [Discussion] Improvement to use Siddhi
>>> str... @ Wed Jul 20, 2016 4:30pm - 5:30pm (IST) (indu...@wso2.com)
>>>
>>> [2]
>>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L108
>>>
>>> [3]
>>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/EventPublisher.java#L311
>>>
>>> Thanks and Regards
>>> --
>>> Indunil Upeksha Rathnayake
>>> Software Engineer | WSO2 Inc
>>> Emailindu...@wso2.com
>>> Mobile   0772182255
>>>
>>>
>>>
>>
>>
>> --
>>
>> *S. Suhothayan*
>> Associate Director / Architect & Team Lead of WSO2 Complex Event
>> Processor
>> *WSO2 Inc. *http://wso2.com
>> * *
>> lean . enterprise . middleware
>>
>>
>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog:
>> http://suhothayan.blogspot.com/ twitter:
>> http://twitter.com/suhothayan  | linked-in:
>> http://lk.linkedin.com/in/suhothayan *
>>
>
>
>
> --
> Indunil Upeksha Rathnayake
> Software Engineer | WSO2 Inc
> Emailindu...@wso2.com
> Mobile   0772182255
>



-- 
Thanks & 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Johann Nallathamby]

On Fri, Jul 22, 2016 at 7:58 AM, Sriskandarajah Suhothayan 
wrote:

>
>
> On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi,
>>
>> Please find the meeting notes in [1].  I have following considerations
>> regarding the improvements we have discussed.
>>
>> (1) Even though we have configured to load the email template from
>> EventPublisher(analytics side), the placeholder values has to be sent as
>> meta data/correlation data/payload data/arbitrary data, since in analytics
>> side, the user claim values are not getting from the user store.
>> In order to send the placeholder values from IS side, anyway we have to
>> load the email template and retrieve the placeholders. So as I have
>> understood, for email notifications, it's not needed to use the email
>> template loading part in analytics, since it'll be a redundant task. (Refer
>> [2])
>>
>
> Here we can set the claim values as arbitrary data, and the notification
> specific details as the meta, correlation & payload data.
> Then we can use the template loading only at the analytics side.
>
>
>> (2) The email templates has to be changed as follows.
>> i) if the value will be provided in an arbitrary data map, the
>> placeholder should be with a prefix "arbitrary_"
>> (ex:{{arbitrary_givenname}})
>>
> ii) if the value will be provided in an meta data map, the placeholder
>> should be changed as {{...}} (ex:{{givenname}})
>>
>> No we should not use "arbitrary_" for any cases, its internal information
> and the names should not have "arbitrary_" even if its in arbitrary data
> map or otherwise.
>
> (3) Only Text OutputMapping Content can be filled from a value in an
>> arbitrary data map using prefix "arbitrary_" .  It's not possible to use a
>> value of an arbitrary data map, in a Dynamic adapter properties, only a
>> value from a meta data/correlation data/payload data map can be used. I
>> think that need to be extended to use even an arbitrary value as a dynamic
>> adapter property.(Refer [3])
>>
>
> @Gobi can you please fix this if that's the case.
>
>
>>
>> (4) The default stream definitions and publisher definitions has to be
>> deployed on super tenant as well as other tenants as well. And when a new
>> tenant is added, those streams and publishers has to be deployed for that
>> particular tenant as well.
>>
>> We can have a tenant creation handler to do this copying during that
> tenant creation time. WDYT?
>

+1. We should write a TenantMgtListener and create these items when a
tenant is created.


>
> Really appreciate your ideas/suggestions regarding the above mentioned
>> concerns.
>>
>> [1] Invitation: [Architecture] [Discussion] Improvement to use Siddhi
>> str... @ Wed Jul 20, 2016 4:30pm - 5:30pm (IST) (indu...@wso2.com)
>>
>> [2]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L108
>>
>> [3]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/EventPublisher.java#L311
>>
>> Thanks and Regards
>> --
>> Indunil Upeksha Rathnayake
>> Software Engineer | WSO2 Inc
>> Emailindu...@wso2.com
>> Mobile   0772182255
>>
>>
>>
>
>
> --
>
> *S. Suhothayan*
> Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
> *WSO2 Inc. *http://wso2.com
> * *
> lean . enterprise . middleware
>
>
> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog:
> http://suhothayan.blogspot.com/ twitter:
> http://twitter.com/suhothayan  | linked-in:
> http://lk.linkedin.com/in/suhothayan *
>



-- 
Thanks & Regards,

*Johann Dilantha Nallathamby*
Technical Lead & Product Lead of WSO2 Identity Server
Governance Technologies Team
WSO2, Inc.
lean.enterprise.middleware

Mobile - *+9476950*
Blog - *http://nallaa.wordpress.com *
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Indunil Upeksha Rathnayake]

Hi,

On Fri, Jul 22, 2016 at 12:28 PM, Sriskandarajah Suhothayan 
wrote:

>
>
> On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi,
>>
>> Please find the meeting notes in [1].  I have following considerations
>> regarding the improvements we have discussed.
>>
>> (1) Even though we have configured to load the email template from
>> EventPublisher(analytics side), the placeholder values has to be sent as
>> meta data/correlation data/payload data/arbitrary data, since in analytics
>> side, the user claim values are not getting from the user store.
>> In order to send the placeholder values from IS side, anyway we have to
>> load the email template and retrieve the placeholders. So as I have
>> understood, for email notifications, it's not needed to use the email
>> template loading part in analytics, since it'll be a redundant task. (Refer
>> [2])
>>
>
> Here we can set the claim values as arbitrary data, and the notification
> specific details as the meta, correlation & payload data.
> Then we can use the template loading only at the analytics side.
>
In this case, from IS side, without parsing only the user claims needed for
a particular email template(i.e.user claim values for the placeholders in
email template), we have to pass all the user claims as arbitrary data
values. In that case there's no need for loading the template from the
registry in IS side. So that in analytics side, all the values needed for
filling out the template will be there. Will check on that.

>
>
>> (2) The email templates has to be changed as follows.
>> i) if the value will be provided in an arbitrary data map, the
>> placeholder should be with a prefix "arbitrary_"
>> (ex:{{arbitrary_givenname}})
>>
> ii) if the value will be provided in an meta data map, the placeholder
>> should be changed as {{...}} (ex:{{givenname}})
>>
>> No we should not use "arbitrary_" for any cases, its internal information
> and the names should not have "arbitrary_" even if its in arbitrary data
> map or otherwise.
>
> (3) Only Text OutputMapping Content can be filled from a value in an
>> arbitrary data map using prefix "arbitrary_" .  It's not possible to use a
>> value of an arbitrary data map, in a Dynamic adapter properties, only a
>> value from a meta data/correlation data/payload data map can be used. I
>> think that need to be extended to use even an arbitrary value as a dynamic
>> adapter property.(Refer [3])
>>
>
> @Gobi can you please fix this if that's the case.
>
>
>>
>> (4) The default stream definitions and publisher definitions has to be
>> deployed on super tenant as well as other tenants as well. And when a new
>> tenant is added, those streams and publishers has to be deployed for that
>> particular tenant as well.
>>
>> We can have a tenant creation handler to do this copying during that
> tenant creation time. WDYT?
>
> Really appreciate your ideas/suggestions regarding the above mentioned
>> concerns.
>>
>> [1] Invitation: [Architecture] [Discussion] Improvement to use Siddhi
>> str... @ Wed Jul 20, 2016 4:30pm - 5:30pm (IST) (indu...@wso2.com)
>>
>> [2]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L108
>>
>> [3]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/EventPublisher.java#L311
>>
>> Thanks and Regards
>> --
>> Indunil Upeksha Rathnayake
>> Software Engineer | WSO2 Inc
>> Emailindu...@wso2.com
>> Mobile   0772182255
>>
>>
>>
>
>
> --
>
> *S. Suhothayan*
> Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
> *WSO2 Inc. *http://wso2.com
> * *
> lean . enterprise . middleware
>
>
> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog:
> http://suhothayan.blogspot.com/ twitter:
> http://twitter.com/suhothayan  | linked-in:
> http://lk.linkedin.com/in/suhothayan *
>



-- 
Indunil Upeksha Rathnayake
Software Engineer | WSO2 Inc
Emailindu...@wso2.com
Mobile   0772182255
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Gobinath Loganathan]

Hi Suho and all,

The problem has been identified and fixed with the new commit [1].
According to this commit, the following changes have been done:

(1) No need to use arbitrary_ prefix anywhere.
(2) The publisher will first check for payload data, if not available it
will check for the arbitrary data. Therefore use the arbitrary data map
keys as they are. [2][3]
(3) Arbitrary data can be used in dynamic properties. [3]

[1]
https://github.com/wso2/carbon-analytics-common/commit/f1d5cafe92e9b71b884a9a4838e2cc446cf83685

[2]
https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L168

[3]
https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/EventPublisher.java#L339

On Fri, Jul 22, 2016 at 12:28 PM, Sriskandarajah Suhothayan 
wrote:

>
>
> On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi,
>>
>> Please find the meeting notes in [1].  I have following considerations
>> regarding the improvements we have discussed.
>>
>> (1) Even though we have configured to load the email template from
>> EventPublisher(analytics side), the placeholder values has to be sent as
>> meta data/correlation data/payload data/arbitrary data, since in analytics
>> side, the user claim values are not getting from the user store.
>> In order to send the placeholder values from IS side, anyway we have to
>> load the email template and retrieve the placeholders. So as I have
>> understood, for email notifications, it's not needed to use the email
>> template loading part in analytics, since it'll be a redundant task. (Refer
>> [2])
>>
>
> Here we can set the claim values as arbitrary data, and the notification
> specific details as the meta, correlation & payload data.
> Then we can use the template loading only at the analytics side.
>
>
>> (2) The email templates has to be changed as follows.
>> i) if the value will be provided in an arbitrary data map, the
>> placeholder should be with a prefix "arbitrary_"
>> (ex:{{arbitrary_givenname}})
>>
> ii) if the value will be provided in an meta data map, the placeholder
>> should be changed as {{...}} (ex:{{givenname}})
>>
>> No we should not use "arbitrary_" for any cases, its internal information
> and the names should not have "arbitrary_" even if its in arbitrary data
> map or otherwise.
>
> (3) Only Text OutputMapping Content can be filled from a value in an
>> arbitrary data map using prefix "arbitrary_" .  It's not possible to use a
>> value of an arbitrary data map, in a Dynamic adapter properties, only a
>> value from a meta data/correlation data/payload data map can be used. I
>> think that need to be extended to use even an arbitrary value as a dynamic
>> adapter property.(Refer [3])
>>
>
> @Gobi can you please fix this if that's the case.
>
>
>>
>> (4) The default stream definitions and publisher definitions has to be
>> deployed on super tenant as well as other tenants as well. And when a new
>> tenant is added, those streams and publishers has to be deployed for that
>> particular tenant as well.
>>
>> We can have a tenant creation handler to do this copying during that
> tenant creation time. WDYT?
>
> Really appreciate your ideas/suggestions regarding the above mentioned
>> concerns.
>>
>> [1] Invitation: [Architecture] [Discussion] Improvement to use Siddhi
>> str... @ Wed Jul 20, 2016 4:30pm - 5:30pm (IST) (indu...@wso2.com)
>>
>> [2]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L108
>>
>> [3]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/EventPublisher.java#L311
>>
>> Thanks and Regards
>> --
>> Indunil Upeksha Rathnayake
>> Software Engineer | WSO2 Inc
>> Emailindu...@wso2.com
>> Mobile   0772182255
>>
>>
>>
>
>
> --
>
> *S. Suhothayan*
> Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
> *WSO2 Inc. *http://wso2.com
> * *
> lean . enterprise . middleware
>
>
> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog:
> http://suhothayan.blogspot.com/ twitter:
> http://twitter.com/suhothayan  | linked-in:
> http://lk.linkedin.com/in/suhothayan *
>



-- 
*Gobinath **Loganathan*
Software Engineer, WSO2 Inc. http://wso2.com

Email  : gobin...@wso2.com
Mobile : (+94) 770 780 210
Blog: javahelps.com 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Sriskandarajah Suhothayan]

On Fri, Jul 22, 2016 at 12:00 PM, Indunil Upeksha Rathnayake <
indu...@wso2.com> wrote:

> Hi,
>
> Please find the meeting notes in [1].  I have following considerations
> regarding the improvements we have discussed.
>
> (1) Even though we have configured to load the email template from
> EventPublisher(analytics side), the placeholder values has to be sent as
> meta data/correlation data/payload data/arbitrary data, since in analytics
> side, the user claim values are not getting from the user store.
> In order to send the placeholder values from IS side, anyway we have to
> load the email template and retrieve the placeholders. So as I have
> understood, for email notifications, it's not needed to use the email
> template loading part in analytics, since it'll be a redundant task. (Refer
> [2])
>

Here we can set the claim values as arbitrary data, and the notification
specific details as the meta, correlation & payload data.
Then we can use the template loading only at the analytics side.


> (2) The email templates has to be changed as follows.
> i) if the value will be provided in an arbitrary data map, the
> placeholder should be with a prefix "arbitrary_"
> (ex:{{arbitrary_givenname}})
>
ii) if the value will be provided in an meta data map, the placeholder
> should be changed as {{...}} (ex:{{givenname}})
>
> No we should not use "arbitrary_" for any cases, its internal information
and the names should not have "arbitrary_" even if its in arbitrary data
map or otherwise.

(3) Only Text OutputMapping Content can be filled from a value in an
> arbitrary data map using prefix "arbitrary_" .  It's not possible to use a
> value of an arbitrary data map, in a Dynamic adapter properties, only a
> value from a meta data/correlation data/payload data map can be used. I
> think that need to be extended to use even an arbitrary value as a dynamic
> adapter property.(Refer [3])
>

@Gobi can you please fix this if that's the case.


>
> (4) The default stream definitions and publisher definitions has to be
> deployed on super tenant as well as other tenants as well. And when a new
> tenant is added, those streams and publishers has to be deployed for that
> particular tenant as well.
>
> We can have a tenant creation handler to do this copying during that
tenant creation time. WDYT?

Really appreciate your ideas/suggestions regarding the above mentioned
> concerns.
>
> [1] Invitation: [Architecture] [Discussion] Improvement to use Siddhi
> str... @ Wed Jul 20, 2016 4:30pm - 5:30pm (IST) (indu...@wso2.com)
>
> [2]
> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L108
>
> [3]
> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/EventPublisher.java#L311
>
> Thanks and Regards
> --
> Indunil Upeksha Rathnayake
> Software Engineer | WSO2 Inc
> Emailindu...@wso2.com
> Mobile   0772182255
>
>
>


-- 

*S. Suhothayan*
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
*WSO2 Inc. *http://wso2.com
* *
lean . enterprise . middleware


*cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan
 | linked-in:
http://lk.linkedin.com/in/suhothayan *
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Indunil Upeksha Rathnayake]

Hi,

Thanks for the response. I have arranged a meeting from 4.30pm - 5.30pm
today(20/7/2016).

Best Regards


On Tue, Jul 19, 2016 at 11:58 AM, Sriskandarajah Suhothayan 
wrote:

> Since Option 2 is now possible I think you can move to it now. The
> advantage is, with this approach you are not restricted to emails and you
> can now use SOAP, REST and other adopters to trigger some actions based on
> notifications, which will make IS much more powerful than just sending
> emails.
>
> I'm available from 2.30 pm at PG.
>
> Regards
> Suho
>
> On Tue, Jul 19, 2016 at 11:17 AM, Johann Nallathamby 
> wrote:
>
>> Hi Suho,
>>
>> On Mon, Jul 18, 2016 at 11:44 PM, Sriskandarajah Suhothayan <
>> s...@wso2.com> wrote:
>>
>>> Hi
>>>
>>> Based on the request of IS team we have recently added support for
>>> loading template files from the registry.
>>> I think with this feature we can do the mapping at Event Publisher side,
>>> then IS can send only the core data for the notification. I think building
>>> the whole message at IS is too much customization for emails.
>>>
>>
>> As discussed previously both methods should work.
>>
>> Replacing placeholder with data in the arbitrary data map was in the
>> master at the time and now it should have been release AFAIU. This is what
>> Indunil was trying.
>>
>> And also you guys have added the support to pick registry templates based
>> on some place holder values in the registry path. What we discussed was to
>> send the 'locale' value as a stream attribute for our use case. If this
>> approach works this is also fine for us.
>>
>> We tried with option1 just to get something working quickly.
>>
>>
>>>
>>> Please set up a meeting so we can discuss the possible ways to
>>> implementing this.
>>>
>>> Regards
>>> Suho
>>>
>>> On Mon, Jul 18, 2016 at 5:52 PM, Indunil Upeksha Rathnayake <
>>> indu...@wso2.com> wrote:
>>>
 Hi,

 We are trying to do some improvements to the notification sending
 module where we have integrated analytics common features in IS, in order
 to send several notifications (ex:Email, SMS).

 Current implementation is in [1], there only the email notification was
 focused where we are directly publishing to the EmailEventAdapter.

 Now we are trying to send notifications via publishing an event to the
 Event stream without directly calling an Output Adapter. The approach we
 have taken is as follows.









 *1) In server start up following will be created.i) A stream for each
 and every notification type including the necessary attributes.Ex:
 Email Notification - a Stream with the subject, body and footer as
 attributesii) Event Publishers, registered for each and every stream in the
 required Output event adapter type. Ex: Email Notification - event
 Publisher in email output event adapter type.2) Publishing an event to
 EventStreamService, which includes an arbitrary data map with the necessary
 data needed for the specific notification type.  Ex: Email Notification
 - Please find the code segments in [2] for having a better understanding.*

 There in IS side, we are selecting a specific email template and will
 be filled out the place holders before sending the subject, body and footer
 as arbitrary map attributes.

 But even-though we passed an arbitrary data map, when we are sending an
 email from the EmailEventAdapter, it won't filter out the subject, body or
 header from that arbitrary data map.
 As I have understood, if someone pass an event with an arbitrary data
 map, the email body will be set as [3] (Refer [4]), it won't filter out the
 content(Refer [5]).
 Is this has to be worked if we provide *output mappings* for event
 publisher as* {{subject}{body}{footer}}* to convert the event to the
 supported format?

 I have gone through the code [6], where the event data will be passed
 through EventStreamProducer, but there also seems like it's not
 possible to send an email in required format(subject, body and footer).

 Really appreciate your comments/suggestions to understand the correct
 approach to be taken.

 [1]
 https://github.com/wso2-extensions/identity-event-handler-email/blob/master/components/event-handler-email/org.wso2.carbon.identity.event.handler.email/src/main/java/org.wso2.carbon.identity.event.handler.email/handler/EmailEventHandler.java#L164
 [2]
 https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBY00yYVpGelZJNms/view?usp=sharing
 [3]
 https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBNEMtYjJvSFB2emM/view?usp=sharing
 [4]
 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Sriskandarajah Suhothayan]

Since Option 2 is now possible I think you can move to it now. The
advantage is, with this approach you are not restricted to emails and you
can now use SOAP, REST and other adopters to trigger some actions based on
notifications, which will make IS much more powerful than just sending
emails.

I'm available from 2.30 pm at PG.

Regards
Suho

On Tue, Jul 19, 2016 at 11:17 AM, Johann Nallathamby 
wrote:

> Hi Suho,
>
> On Mon, Jul 18, 2016 at 11:44 PM, Sriskandarajah Suhothayan  > wrote:
>
>> Hi
>>
>> Based on the request of IS team we have recently added support for
>> loading template files from the registry.
>> I think with this feature we can do the mapping at Event Publisher side,
>> then IS can send only the core data for the notification. I think building
>> the whole message at IS is too much customization for emails.
>>
>
> As discussed previously both methods should work.
>
> Replacing placeholder with data in the arbitrary data map was in the
> master at the time and now it should have been release AFAIU. This is what
> Indunil was trying.
>
> And also you guys have added the support to pick registry templates based
> on some place holder values in the registry path. What we discussed was to
> send the 'locale' value as a stream attribute for our use case. If this
> approach works this is also fine for us.
>
> We tried with option1 just to get something working quickly.
>
>
>>
>> Please set up a meeting so we can discuss the possible ways to
>> implementing this.
>>
>> Regards
>> Suho
>>
>> On Mon, Jul 18, 2016 at 5:52 PM, Indunil Upeksha Rathnayake <
>> indu...@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> We are trying to do some improvements to the notification sending module
>>> where we have integrated analytics common features in IS, in order to send
>>> several notifications (ex:Email, SMS).
>>>
>>> Current implementation is in [1], there only the email notification was
>>> focused where we are directly publishing to the EmailEventAdapter.
>>>
>>> Now we are trying to send notifications via publishing an event to the
>>> Event stream without directly calling an Output Adapter. The approach we
>>> have taken is as follows.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *1) In server start up following will be created.i) A stream for each
>>> and every notification type including the necessary attributes.Ex:
>>> Email Notification - a Stream with the subject, body and footer as
>>> attributesii) Event Publishers, registered for each and every stream in the
>>> required Output event adapter type. Ex: Email Notification - event
>>> Publisher in email output event adapter type.2) Publishing an event to
>>> EventStreamService, which includes an arbitrary data map with the necessary
>>> data needed for the specific notification type.  Ex: Email Notification
>>> - Please find the code segments in [2] for having a better understanding.*
>>>
>>> There in IS side, we are selecting a specific email template and will be
>>> filled out the place holders before sending the subject, body and footer as
>>> arbitrary map attributes.
>>>
>>> But even-though we passed an arbitrary data map, when we are sending an
>>> email from the EmailEventAdapter, it won't filter out the subject, body or
>>> header from that arbitrary data map.
>>> As I have understood, if someone pass an event with an arbitrary data
>>> map, the email body will be set as [3] (Refer [4]), it won't filter out the
>>> content(Refer [5]).
>>> Is this has to be worked if we provide *output mappings* for event
>>> publisher as* {{subject}{body}{footer}}* to convert the event to the
>>> supported format?
>>>
>>> I have gone through the code [6], where the event data will be passed
>>> through EventStreamProducer, but there also seems like it's not
>>> possible to send an email in required format(subject, body and footer).
>>>
>>> Really appreciate your comments/suggestions to understand the correct
>>> approach to be taken.
>>>
>>> [1]
>>> https://github.com/wso2-extensions/identity-event-handler-email/blob/master/components/event-handler-email/org.wso2.carbon.identity.event.handler.email/src/main/java/org.wso2.carbon.identity.event.handler.email/handler/EmailEventHandler.java#L164
>>> [2]
>>> https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBY00yYVpGelZJNms/view?usp=sharing
>>> [3]
>>> https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBNEMtYjJvSFB2emM/view?usp=sharing
>>> [4]
>>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L139
>>> [5]
>>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/event-output-adapters/org.wso2.carbon.event.output.adapter.email/src/main/java/org/wso2/carbon/event/output/adapter/email/EmailEventAdapter.java#L233
>>> [6]
>>> 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Johann Nallathamby]

Hi Suho,

On Mon, Jul 18, 2016 at 11:44 PM, Sriskandarajah Suhothayan 
wrote:

> Hi
>
> Based on the request of IS team we have recently added support for loading
> template files from the registry.
> I think with this feature we can do the mapping at Event Publisher side,
> then IS can send only the core data for the notification. I think building
> the whole message at IS is too much customization for emails.
>

As discussed previously both methods should work.

Replacing placeholder with data in the arbitrary data map was in the master
at the time and now it should have been release AFAIU. This is what Indunil
was trying.

And also you guys have added the support to pick registry templates based
on some place holder values in the registry path. What we discussed was to
send the 'locale' value as a stream attribute for our use case. If this
approach works this is also fine for us.

We tried with option1 just to get something working quickly.


>
> Please set up a meeting so we can discuss the possible ways to
> implementing this.
>
> Regards
> Suho
>
> On Mon, Jul 18, 2016 at 5:52 PM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi,
>>
>> We are trying to do some improvements to the notification sending module
>> where we have integrated analytics common features in IS, in order to send
>> several notifications (ex:Email, SMS).
>>
>> Current implementation is in [1], there only the email notification was
>> focused where we are directly publishing to the EmailEventAdapter.
>>
>> Now we are trying to send notifications via publishing an event to the
>> Event stream without directly calling an Output Adapter. The approach we
>> have taken is as follows.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *1) In server start up following will be created.i) A stream for each and
>> every notification type including the necessary attributes.Ex: Email
>> Notification - a Stream with the subject, body and footer as attributesii)
>> Event Publishers, registered for each and every stream in the required
>> Output event adapter type. Ex: Email Notification - event Publisher in
>> email output event adapter type.2) Publishing an event to
>> EventStreamService, which includes an arbitrary data map with the necessary
>> data needed for the specific notification type.  Ex: Email Notification
>> - Please find the code segments in [2] for having a better understanding.*
>>
>> There in IS side, we are selecting a specific email template and will be
>> filled out the place holders before sending the subject, body and footer as
>> arbitrary map attributes.
>>
>> But even-though we passed an arbitrary data map, when we are sending an
>> email from the EmailEventAdapter, it won't filter out the subject, body or
>> header from that arbitrary data map.
>> As I have understood, if someone pass an event with an arbitrary data
>> map, the email body will be set as [3] (Refer [4]), it won't filter out the
>> content(Refer [5]).
>> Is this has to be worked if we provide *output mappings* for event
>> publisher as* {{subject}{body}{footer}}* to convert the event to the
>> supported format?
>>
>> I have gone through the code [6], where the event data will be passed
>> through EventStreamProducer, but there also seems like it's not possible
>> to send an email in required format(subject, body and footer).
>>
>> Really appreciate your comments/suggestions to understand the correct
>> approach to be taken.
>>
>> [1]
>> https://github.com/wso2-extensions/identity-event-handler-email/blob/master/components/event-handler-email/org.wso2.carbon.identity.event.handler.email/src/main/java/org.wso2.carbon.identity.event.handler.email/handler/EmailEventHandler.java#L164
>> [2]
>> https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBY00yYVpGelZJNms/view?usp=sharing
>> [3]
>> https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBNEMtYjJvSFB2emM/view?usp=sharing
>> [4]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L139
>> [5]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/event-output-adapters/org.wso2.carbon.event.output.adapter.email/src/main/java/org/wso2/carbon/event/output/adapter/email/EmailEventAdapter.java#L233
>> [6]
>> https://github.com/wso2/carbon-event-processing/blob/master/components/event-simulator/org.wso2.carbon.event.simulator.core/src/main/java/org/wso2/carbon/event/simulator/core/internal/CarbonEventSimulator.java#L183
>>
>> Thanks and Regards
>> --
>> Indunil Upeksha Rathnayake
>> Software Engineer | WSO2 Inc
>> Emailindu...@wso2.com
>> Mobile   0772182255
>>
>
>
>
> --
>
> *S. Suhothayan*
> Technical Lead & Team Lead of WSO2 Complex Event Processor
> *WSO2 Inc. *http://wso2.com
> * *
> lean . enterprise . middleware
>
>
> *cell: (+94) 779 756 757 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Indunil Upeksha Rathnayake]

Hi,

Thanks for the response. As you suggested, It's better to discuss and find
a possible way of implementing this. Shall I arrange a meeting
tomorrow(19/7/2016)? If possible, please let me know an available time slot.

Thanks and Regards

On Mon, Jul 18, 2016 at 11:44 PM, Sriskandarajah Suhothayan 
wrote:

> Hi
>
> Based on the request of IS team we have recently added support for loading
> template files from the registry.
> I think with this feature we can do the mapping at Event Publisher side,
> then IS can send only the core data for the notification. I think building
> the whole message at IS is too much customization for emails.
>
> Please set up a meeting so we can discuss the possible ways to
> implementing this.
>
> Regards
> Suho
>
> On Mon, Jul 18, 2016 at 5:52 PM, Indunil Upeksha Rathnayake <
> indu...@wso2.com> wrote:
>
>> Hi,
>>
>> We are trying to do some improvements to the notification sending module
>> where we have integrated analytics common features in IS, in order to send
>> several notifications (ex:Email, SMS).
>>
>> Current implementation is in [1], there only the email notification was
>> focused where we are directly publishing to the EmailEventAdapter.
>>
>> Now we are trying to send notifications via publishing an event to the
>> Event stream without directly calling an Output Adapter. The approach we
>> have taken is as follows.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *1) In server start up following will be created.i) A stream for each and
>> every notification type including the necessary attributes.Ex: Email
>> Notification - a Stream with the subject, body and footer as attributesii)
>> Event Publishers, registered for each and every stream in the required
>> Output event adapter type. Ex: Email Notification - event Publisher in
>> email output event adapter type.2) Publishing an event to
>> EventStreamService, which includes an arbitrary data map with the necessary
>> data needed for the specific notification type.  Ex: Email Notification
>> - Please find the code segments in [2] for having a better understanding.*
>>
>> There in IS side, we are selecting a specific email template and will be
>> filled out the place holders before sending the subject, body and footer as
>> arbitrary map attributes.
>>
>> But even-though we passed an arbitrary data map, when we are sending an
>> email from the EmailEventAdapter, it won't filter out the subject, body or
>> header from that arbitrary data map.
>> As I have understood, if someone pass an event with an arbitrary data
>> map, the email body will be set as [3] (Refer [4]), it won't filter out the
>> content(Refer [5]).
>> Is this has to be worked if we provide *output mappings* for event
>> publisher as* {{subject}{body}{footer}}* to convert the event to the
>> supported format?
>>
>> I have gone through the code [6], where the event data will be passed
>> through EventStreamProducer, but there also seems like it's not possible
>> to send an email in required format(subject, body and footer).
>>
>> Really appreciate your comments/suggestions to understand the correct
>> approach to be taken.
>>
>> [1]
>> https://github.com/wso2-extensions/identity-event-handler-email/blob/master/components/event-handler-email/org.wso2.carbon.identity.event.handler.email/src/main/java/org.wso2.carbon.identity.event.handler.email/handler/EmailEventHandler.java#L164
>> [2]
>> https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBY00yYVpGelZJNms/view?usp=sharing
>> [3]
>> https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBNEMtYjJvSFB2emM/view?usp=sharing
>> [4]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L139
>> [5]
>> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/event-output-adapters/org.wso2.carbon.event.output.adapter.email/src/main/java/org/wso2/carbon/event/output/adapter/email/EmailEventAdapter.java#L233
>> [6]
>> https://github.com/wso2/carbon-event-processing/blob/master/components/event-simulator/org.wso2.carbon.event.simulator.core/src/main/java/org/wso2/carbon/event/simulator/core/internal/CarbonEventSimulator.java#L183
>>
>> Thanks and Regards
>> --
>> Indunil Upeksha Rathnayake
>> Software Engineer | WSO2 Inc
>> Emailindu...@wso2.com
>> Mobile   0772182255
>>
>
>
>
> --
>
> *S. Suhothayan*
> Technical Lead & Team Lead of WSO2 Complex Event Processor
> *WSO2 Inc. *http://wso2.com
> * *
> lean . enterprise . middleware
>
>
> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog:
> http://suhothayan.blogspot.com/ twitter:
> http://twitter.com/suhothayan  | linked-in:
> http://lk.linkedin.com/in/suhothayan *
>



-- 
Indunil Upeksha Rathnayake
Software Engineer | WSO2 

Re: [Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Sriskandarajah Suhothayan]

Hi

Based on the request of IS team we have recently added support for loading
template files from the registry.
I think with this feature we can do the mapping at Event Publisher side,
then IS can send only the core data for the notification. I think building
the whole message at IS is too much customization for emails.

Please set up a meeting so we can discuss the possible ways to implementing
this.

Regards
Suho

On Mon, Jul 18, 2016 at 5:52 PM, Indunil Upeksha Rathnayake <
indu...@wso2.com> wrote:

> Hi,
>
> We are trying to do some improvements to the notification sending module
> where we have integrated analytics common features in IS, in order to send
> several notifications (ex:Email, SMS).
>
> Current implementation is in [1], there only the email notification was
> focused where we are directly publishing to the EmailEventAdapter.
>
> Now we are trying to send notifications via publishing an event to the
> Event stream without directly calling an Output Adapter. The approach we
> have taken is as follows.
>
>
>
>
>
>
>
>
>
> *1) In server start up following will be created.i) A stream for each and
> every notification type including the necessary attributes.Ex: Email
> Notification - a Stream with the subject, body and footer as attributesii)
> Event Publishers, registered for each and every stream in the required
> Output event adapter type. Ex: Email Notification - event Publisher in
> email output event adapter type.2) Publishing an event to
> EventStreamService, which includes an arbitrary data map with the necessary
> data needed for the specific notification type.  Ex: Email Notification
> - Please find the code segments in [2] for having a better understanding.*
>
> There in IS side, we are selecting a specific email template and will be
> filled out the place holders before sending the subject, body and footer as
> arbitrary map attributes.
>
> But even-though we passed an arbitrary data map, when we are sending an
> email from the EmailEventAdapter, it won't filter out the subject, body or
> header from that arbitrary data map.
> As I have understood, if someone pass an event with an arbitrary data map,
> the email body will be set as [3] (Refer [4]), it won't filter out the
> content(Refer [5]).
> Is this has to be worked if we provide *output mappings* for event
> publisher as* {{subject}{body}{footer}}* to convert the event to the
> supported format?
>
> I have gone through the code [6], where the event data will be passed
> through EventStreamProducer, but there also seems like it's not possible
> to send an email in required format(subject, body and footer).
>
> Really appreciate your comments/suggestions to understand the correct
> approach to be taken.
>
> [1]
> https://github.com/wso2-extensions/identity-event-handler-email/blob/master/components/event-handler-email/org.wso2.carbon.identity.event.handler.email/src/main/java/org.wso2.carbon.identity.event.handler.email/handler/EmailEventHandler.java#L164
> [2]
> https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBY00yYVpGelZJNms/view?usp=sharing
> [3]
> https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBNEMtYjJvSFB2emM/view?usp=sharing
> [4]
> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L139
> [5]
> https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/event-output-adapters/org.wso2.carbon.event.output.adapter.email/src/main/java/org/wso2/carbon/event/output/adapter/email/EmailEventAdapter.java#L233
> [6]
> https://github.com/wso2/carbon-event-processing/blob/master/components/event-simulator/org.wso2.carbon.event.simulator.core/src/main/java/org/wso2/carbon/event/simulator/core/internal/CarbonEventSimulator.java#L183
>
> Thanks and Regards
> --
> Indunil Upeksha Rathnayake
> Software Engineer | WSO2 Inc
> Emailindu...@wso2.com
> Mobile   0772182255
>



-- 

*S. Suhothayan*
Technical Lead & Team Lead of WSO2 Complex Event Processor
*WSO2 Inc. *http://wso2.com
* *
lean . enterprise . middleware


*cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan
 | linked-in:
http://lk.linkedin.com/in/suhothayan *
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] [Architecture] [IS] [Analytics] Improvement to use Siddhi streams to send notifications

[Indunil Upeksha Rathnayake]

Hi,

We are trying to do some improvements to the notification sending module
where we have integrated analytics common features in IS, in order to send
several notifications (ex:Email, SMS).

Current implementation is in [1], there only the email notification was
focused where we are directly publishing to the EmailEventAdapter.

Now we are trying to send notifications via publishing an event to the
Event stream without directly calling an Output Adapter. The approach we
have taken is as follows.









*1) In server start up following will be created.i) A stream for each and
every notification type including the necessary attributes.Ex: Email
Notification - a Stream with the subject, body and footer as attributesii)
Event Publishers, registered for each and every stream in the required
Output event adapter type. Ex: Email Notification - event Publisher in
email output event adapter type.2) Publishing an event to
EventStreamService, which includes an arbitrary data map with the necessary
data needed for the specific notification type.  Ex: Email Notification
- Please find the code segments in [2] for having a better understanding.*

There in IS side, we are selecting a specific email template and will be
filled out the place holders before sending the subject, body and footer as
arbitrary map attributes.

But even-though we passed an arbitrary data map, when we are sending an
email from the EmailEventAdapter, it won't filter out the subject, body or
header from that arbitrary data map.
As I have understood, if someone pass an event with an arbitrary data map,
the email body will be set as [3] (Refer [4]), it won't filter out the
content(Refer [5]).
Is this has to be worked if we provide *output mappings* for event
publisher as* {{subject}{body}{footer}}* to convert the event to the
supported format?

I have gone through the code [6], where the event data will be passed
through EventStreamProducer, but there also seems like it's not possible to
send an email in required format(subject, body and footer).

Really appreciate your comments/suggestions to understand the correct
approach to be taken.

[1]
https://github.com/wso2-extensions/identity-event-handler-email/blob/master/components/event-handler-email/org.wso2.carbon.identity.event.handler.email/src/main/java/org.wso2.carbon.identity.event.handler.email/handler/EmailEventHandler.java#L164
[2]
https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBY00yYVpGelZJNms/view?usp=sharing
[3]
https://drive.google.com/a/wso2.com/file/d/0Bz_EQkE2mOgBNEMtYjJvSFB2emM/view?usp=sharing
[4]
https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/org.wso2.carbon.event.publisher.core/src/main/java/org/wso2/carbon/event/publisher/core/internal/type/text/TextOutputMapper.java#L139
[5]
https://github.com/wso2/carbon-analytics-common/blob/master/components/event-publisher/event-output-adapters/org.wso2.carbon.event.output.adapter.email/src/main/java/org/wso2/carbon/event/output/adapter/email/EmailEventAdapter.java#L233
[6]
https://github.com/wso2/carbon-event-processing/blob/master/components/event-simulator/org.wso2.carbon.event.simulator.core/src/main/java/org/wso2/carbon/event/simulator/core/internal/CarbonEventSimulator.java#L183

Thanks and Regards
-- 
Indunil Upeksha Rathnayake
Software Engineer | WSO2 Inc
Emailindu...@wso2.com
Mobile   0772182255
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev