[Dev] Fwd: GSOC2016: [ML][CEP] [SAMOA]Predictive analytic with online data for WSO2 Machine Learner-Samoa Integration

[Mahesh Dananjaya]

-- Forwarded message --
From: Mahesh Dananjaya 
Date: Sun, Jul 17, 2016 at 11:18 AM
Subject: Re: GSOC2016: [ML][CEP] [SAMOA]Predictive analytic with online
data for WSO2 Machine Learner-Samoa Integration
To: Maheshakya Wijewardena 


Hi Maheshakaya,
just need a little help. In Samoa when we want to run a class what is does
it used this commands [1],
1. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar
"ClusteringEvaluation"
2. bin/samoa storm target/SAMOA-Storm-0.0.1-SNAPSHOT.jar
"PrequentialEvaluation -d /tmp/dump.csv -i 100 -f 10 -l
(classifiers.trees.VerticalHoeffdingTree -p 4) -s
(generators.RandomTreeGenerator -c 2 -o 10 -u 10)"

what is does is call a class named LocalDoTask [4] and pass this string as
argument.After that that LocalDoTask call the relevent Tasks such as
ClusteringEvaluation or PrequentialEvaluation. [2].

Now i have add samoa dependencies to my new maven project, where i used
original samoa source to write examples and test then earlier.Now i want to
push them into my new java project with samoa dependencies. I added
dependency and it was built fine. Now i am calling my local DoTask.java [3]
file as same as i did with samoa with,
java -cp target/streaming-1.0-SNAPSHOT.jar org.gsoc.samoa.streaming.DoTask
"org.gsoc.samoa.streaming.ClusteringEvaluation"
But seems to be i am incorrect in some place.
Error: A JNI error has occurred, please check your installation and try
again
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/samoa/topology/ComponentFactory
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at
sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException:
org.apache.samoa.topology.ComponentFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more


can i actually call the Task like this.

BR,
Mahesh.

[1]
https://samoa.incubator.apache.org/documentation/Prequential-Evaluation-Task.html
[2]
https://github.com/apache/incubator-samoa/blob/releases/0.4.0-incubating-RC0/samoa-api/src/main/java/org/apache/samoa/tasks/ClusteringEvaluation.java
[3]
https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/samoa/streaming/src/main/java/org/gsoc/samoa/streaming
[4]
https://github.com/apache/incubator-samoa/tree/releases/0.4.0-incubating-RC0/samoa-local/src/main/java/org/apache/samoa


On Thu, Jul 14, 2016 at 3:47 PM, Mahesh Dananjaya  wrote:

> Hi srinath,
> sure.i am working on it.thank you.
> regards,
> Mahesh.
>
> On Thu, Jul 14, 2016 at 11:12 AM, Srinath Perera  wrote:
>
>> Hi Mahesh,
>>
>> Let's focus on getting SAOMA work with CEP. It is OK to be limited to few
>> algorithms.
>>
>> --Srinath
>>
>> On Thu, Jul 14, 2016 at 10:49 AM, Mahesh Dananjaya <
>> dananjayamah...@gmail.com> wrote:
>>
>>> Hi Maheshakya,
>>> I think we can build new tasks [1] like the one in execution plan in cep
>>> with samoa. I will try to build a one.
>>> regards,
>>> Mahesh.
>>> [1]
>>> https://samoa.incubator.apache.org/documentation/Developing-New-Tasks-in-SAMOA.html
>>>
>>>
>>> On Thu, Jul 14, 2016 at 10:35 AM, Mahesh Dananjaya <
>>> dananjayamah...@gmail.com> wrote:
>>>
 Hi Maheshakya,
 I am building and running samoa to see its functionality. In samoa
 still we have limited supports in algorithms. Samoa supports only
 classification and clustering with streams. It also use kind of
 StreamProcessor, like the one we use in StreamProcessor extension.  I was
 getting started with Samoa referring to this page [1]. Then i ran couple of
 examples to identified the flow. Samoa use hadoop framework instead spark
 for distribution. But i am using it in a local mode. When i see the Samoa
 core there is only limited algorithms. IMO if we are going to use Samoa we
 have to limit the functionality and algorithms [2]. When i go to developer
 corner in [3], it seems to be something like CEP extension that we are
 using currenlty. SO in Samoa though the algorihtms are limited, they have
 implemented streaming support for them. Therefore if we integrate it into
 CEP we have to look for how to handle streams and algorithms in Samoa side.
 Is it good for your side to have both hadoop and spark running
 background.thank you.
 regards,
 Mahesh.

 [1] https://samoa.incubator.apache.org/documentation/Home.html

Re: [Dev] [GSOC 16] Initial Project Progress

[Osura Rathnayake]

Hi Imesh,

Sure we shall have a meeting on Monday to review the progress. I will send
a detailed update on the current status by tonight. Sorry for the delay.

Thanks,

On Fri, Jul 15, 2016 at 4:04 PM, Isuru Haththotuwa  wrote:

> Hi Osura,
>
> Can you send a detailed updated on the current status? Shall we have a
> meeting on Monday to review the progress.
>
> On Tue, Jul 12, 2016 at 2:03 PM, Osura Rathnayake 
> wrote:
>
>> Hi Imesh,
>>
>> About dynamically adding members to the load balancer, I will cross check
>> it with auto-scaling. I couldn't look into that from auto-scaling end since
>> I couldn't test it yet.
>>
>> about monitoring, yes we can do a POC on that.
>>
>> thanks,
>>
>> On Tue, Jul 12, 2016 at 12:53 PM, Imesh Gunaratne  wrote:
>>
>>> On Tue, Jul 12, 2016 at 12:09 PM, Osura Rathnayake 
>>> wrote:
>>>
 Hi Imesh,

 About centralized logging, I'm trying to get logs to the Log Analytics
 using few methods supported in azure. We can either parse logs in to
 syslogs and send to the Log Analytics or create custom logs specifying the
 logs location. As you said, logs shouldn't be in .txt extension, I got it
 clarified from a azure blog. Will update you soon after I could resolve it.

 ​Right, thanks for the update!​
>>>
>>>
 No you can't dynamically add VMs to the load balancer. Backend pool,
 where all the VMs reside, should be predefined.

>>>
>>> ​Technically that capability should be there. Otherwise we would not be
>>> able to autoscale a server cluster dynamically.
>>> ​
>>>
>>>
 you can auto-scale using scale sets(I'm still researching about it),
 that's the equivalent of AWS auto scaling group . also you can scale up or
 down a VM if it exceeds a certain parameter like CPU usage, using
 monitoring rules.

 *Monitoring *


 Azure has a native monitoring tool which involves collecting and
 tracking metrics, analyzing log files, defining custom metrics and logging
 generated by specific applications or workloads running in Virtual
 Machines. Azure represents monitored data in a graphical manner using
 charts. Monitoring also facilitates triggering alarms when certain
 conditions are met and also it can be configured to take actions on the met
 conditions. Monitoring is done by the Diagnostic Extension and it has
 following capabilities.

 · Collects and uploads the system performance information from
 the Linux VM to the user's storage table, including diagnostic and syslog
 information.

 · Enables users to customize the data metrics that will be
 collected and uploaded.

 · Enables users to upload specified log files to a designated
 storage table.

 Note: Azure storage tables are a non-relational, key-value-pair,
 storage system suitable for storing massive amounts of unstructured data.


 We can add monitor rules so that when an alert triggers it notifies the
 admins via email. Furthermore we can set to take automated actions. Azure
 automate actions by running runbooks. A runbook is a set of tasks that
 perform some automated process in Azure Automation. We can create our own
 runbooks as well. Available runbooks include,

 · Restart VM

 · Stop VM

 · Remove VM

 · Scale up VM

 · Scale down VM

 When scaling up it sets the virtual machine to the next larger size
 within the size group and when scaling down it sets the virtual machine to
 the next smaller size within the size group.

 More about runbooks and automation [1]

>>>
>>> ​Sounds good, will us be able to do a POC on this?​
>>>
>>>

 *Auto scaling *


 Auto-scaling is the process of dynamically allocating the resources
 required by an application to match performance requirements. Virtual
 machine scale sets are an Azure Compute resource you can use to deploy and
 manage a set of identical VMs. With all VMs configured the same, VM scale
 sets are designed to support true auto-scale  no pre-provisioning of VMs is
 required – and as such makes it easier to build large-scale services
 targeting big compute, big data, and containerized workloads [2].


 Note: I couldn’t practically do this as my azure free account lets me
 have only 4 cores and I have used all of them on my current deployment. I’m
 getting a new azure account from one of my friends in a day so hopefully I
 will do this on it and update you.

>>>
>>> ​Great! Thanks!​
>>>
>>>

 [1]
 https://azure.microsoft.com/en-us/documentation/articles/automation-intro/

 [2]
 https://azure.microsoft.com/en-us/documentation/articles/virtual-machine-scale-sets-overview/


 thanks,

Re: [Dev] Controlling access to jaggery pages by permissions

[Rajkumar Rajaratnam]

Thanks Thanuja for the response. Will follow the current approach.

On Fri, Jul 15, 2016 at 7:35 PM, Thanuja Jayasinghe 
wrote:

>
>
> On Fri, Jul 15, 2016 at 3:56 PM, Rajkumar Rajaratnam 
> wrote:
>
>> Hi,
>>
>> I have a jaggery app with some pages and secured them via SAML SSO with
>> WSO2 IS. So the authentication is implemented, now I have to implement the
>> authorization. I need to control access to these jaggery pages by
>> roles/permissions of the loggedin user. Here is the approach I have
>> followed and I need to validate whether it is okay or there are better
>> ways.
>>
>>1. Created custom permissions under my application service provider
>>(one permission per one feature in my jaggery app)
>>2. When a user access a feature in the jaggery app, I am calling "
>>*isUserAuthorized*" method of "*RemoteAuthorizationManagerService*"
>>admin service to check whether the logged in user is authorized to access
>>the page. I think "isUserAuthorized" method checks whether the given user
>>has any roles with the given permission. So, if it returns true, then I
>>allow the user to access the page.
>>3. I am calling the admin service with basic authentication. Is there
>>any issues with this approach? Do I need to obtain a session cookie and
>>call the admin service using session cookie instead of username/password?
>>What is the recommended approach?
>>
>> Any issues with this approach?
>>
> Since "isUserAuthorized" method of "RemoteAuthorizationManagerService"
> requires "/permission/admin/configure/security" permission, a user
> without this permission will not able to access this service using his
> session cookie. So your current approach is correct.
>
>> Thanks,
>> Raj.
>>
>> --
>> Rajkumar Rajaratnam
>> Committer & PMC Member, Apache Stratos
>> Senior Software Engineer, WSO2
>>
>> Mobile : +94777568639
>>
>
> Thanks,
>
> --
> *Thanuja Lakmal*
> Senior Software Engineer
> WSO2 Inc. http://wso2.com/
> *lean.enterprise.middleware*
> Mobile: +94715979891 +94758009992
>



-- 
Rajkumar Rajaratnam
Committer & PMC Member, Apache Stratos
Senior Software Engineer, WSO2

Mobile : +94777568639
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [VOTE] Release WSO2 API Manager 2.0.0 RC1

[Tharindu Dharmarathna]

*Tested Scenarios*

Throttling - 4 levels in tenant and super tenant.

Black listing

+1

Go ahead and release.

Thanks

On Sat, Jul 16, 2016 at 11:11 AM, Abimaran Kugathasan 
wrote:

> Hi All,
>
> Please find the Reformance Report for above RC pack.
>
> On Sat, Jul 16, 2016 at 5:25 AM, Abimaran Kugathasan 
> wrote:
>
>> Hi Devs,
>>
>> *WSO2 API Manager 2.0.0-RC1 Released*
>>
>> This is the 1st Release Candidate of the WSO2 API Manager 2.0.0
>>
>> Please download, test the product and vote. Vote will be open for 72
>> hours or as needed.
>> Refer to github readme for guides.
>>
>> The distribution is available to download at [1]
>>
>> [1] : https://github.com/wso2/product-apim/releases/tag/v2.0.0-rc1
>>
>> *WSO2 **API Manager Analytics 2.0.0 RC1** Released*
>>
>> Please download, test the product and vote. Vote will be open for 72
>> hours or as needed.
>> Refer to github readme for guides.
>>
>> The distribution is available to download  [2]
>>
>> Fixed Issue in [3], known issues [4]
>>
>> [2] :
>> https://github.com/wso2/analytics-apim/releases/download/v2.0.0-rc1/wso2am-analytics-2.0.0.zip
>> [3] : https://wso2.org/jira/issues/?filter=13138
>> [4] : https://wso2.org/jira/issues/?filter=13139
>> 
>>
>>
>>
>>
>> --
>> Thanks
>> Abimaran Kugathasan
>> Senior Software Engineer
>>
>> Email : abima...@wso2.com
>> Mobile : +94 773922820
>>
>> 
>> 
>>   
>> 
>>
>>
>
>
> --
> Thanks
> Abimaran Kugathasan
> Senior Software Engineer
>
> Email : abima...@wso2.com
> Mobile : +94 773922820
>
> 
> 
>   
> 
>
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 

*Tharindu Dharmarathna*Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware

mobile: *+94779109091*
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [VOTE] WSO2 App Manager 1.2.0 RC1

[Thilini Shanika]

Adding Architecture

On Sat, Jul 16, 2016 at 6:36 AM, Thilini Shanika  wrote:

> Hi Devs,
>
> This is the 1st Release Candidate of the WSO2 App Manager 1.2.0
>
> Please download, test the product and vote. Vote will be open for 72 hours
> or as needed.
> Refer to github readme for guides.
>
> The distribution is available to download at [1]
>
> [1] :
> https://github.com/wso2/product-app-manager/releases/download/v1.2.0-RC1/wso2appm-1.2.0.zip
>
> Fixed Issue in [2], known issues [3], tag to be vote upon [4]
>
> [2] : https://wso2.org/jira/issues/?filter=13104
> [3] : https://wso2.org/jira/browse/WSFRUBY-72?filter=13140
> [4] : https://github.com/wso2/carbon-appmgt/tree/v1.2.0-RC1
>
> Please vote as follows.
> [+] Stable - go ahead and release
> [-]  Broken - do not release (please explain why)
>
> Thanks,
> App Manager Team
>
> --
> Thilini Shanika
> Senior Software Engineer
> WSO2, Inc.; http://wso2.com
> 20, Palmgrove Avenue, Colombo 3
>
> E-mail: tgtshan...@gmail.com
>
>


-- 
Thilini Shanika
Senior Software Engineer
WSO2, Inc.; http://wso2.com
20, Palmgrove Avenue, Colombo 3

E-mail: tgtshan...@gmail.com
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] WSO2 DSS 3.5.1 Beta Released

[Madhawa Gunasekara]

Hi All,

*WSO2 Data Services Server 3.5.1 Beta Released*

WSO2 DSS Team is pleased to announce the WSO2 DSS 3.5.1 - Beta release.

The distribution is available to download at [1]. It contains improvements
and bug fixes [2]. Javadocs can be found in [3].

[1] - https://github.com/wso2/product-dss/releases/tag/v3.5.1-beta1
[2] -
https://wso2.org/jira/secure/ReleaseNote.jspa?projectId=10090=12334
[3] - https://svn.wso2.org/repos/wso2/scratch/DSS/3.5.1/beta1/index.html



*WSO2 DSS Tooling 3.5.1 - Beta Released*

The distribution is available to download at [4].

[4] -
https://github.com/wso2/devstudio-tooling-dss/releases/tag/v3.5.1-beta1



*How To Contribute *
Your feedback is most welcome! Mailing ListsJoin our mailing list and
correspond with the developers directly.

   - Developer List : dev@wso2.org | Subscribe  | Mail
   Archive 
   - User forum : StackOverflow
   

Reporting Issues
Issues/bugs that you may encounter could be reported via the WSO2 DSS JIRA
[5]. Please mark the affected version as 3.5.1-beta

[5] - https://wso2.org/jira/browse/DS

Thank you!

~ WSO2 Data Services Server Team ~
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev