Re: [Architecture] C5 Modifying OSGi bundle deployment logic to support temporary patches (log patches, etc)
> > > Yes, I agree that this will be an empty folder. The reason for my > suggestion is that unlike the other components in the lib folder, this temp > jar will be of the same version as that in the plugins (since this is a > temp jar, IMO we do not need to change the version). Also allowing the jars > in the lib folder to overwrite the plugins folder IMO is wrong because lib > is the place for the customer to add his components and it should not > overwrite the server components, i.e., in the plugins. IMO having this in a > separate location would be clean, since then we give that particular > directory higher precedence than the plugins folder and this folder should > be used for temp purpose only. > +1 for this. Since the word is patch abondoned from c5, how about creating a folder called information-collector or log-collector? Thanks, Lakshman > > >> >> Thanks, >> Jayanga. >> >> *Jayanga Dissanayake* >> Associate Technical Lead >> WSO2 Inc. - http://wso2.com/ >> lean . enterprise . middleware >> email: jaya...@wso2.com >> mobile: +94772207259 <+94%2077%20220%207259> >> <http://wso2.com/signature> >> >> On Mon, May 22, 2017 at 10:52 AM, Niranjan Karunanandham < >> niran...@wso2.com> wrote: >> >>> Hi Jayanga, >>> >>> IMO we should also take into consideration of apply fixes other than >>> just log patches. In C5 onwards, the current approach is to use WUM to >>> deliver all updates. But there can be scenarios where we would need to >>> provide an immediate solution or say a Pre-QA fix. IMO we would need to >>> address this here. >>> >>> IMO applying the temp fix in lib folder is not correct. IMO the Lib >>> folder is used for addition components that are required by the server, >>> i.e., components which are not provided by wso2. IMO it would be better to >>> have a separate folder for this. In carbon 4.4.X, we had a separate folder >>> called patches. >>> >>> Regards, >>> Nira >>> >>> On Fri, May 19, 2017 at 10:46 AM, Jayanga Dissanayake <jaya...@wso2.com> >>> wrote: >>> >>>> Hi Vidura, >>>> >>>> Adding a separate command line would easily lead to more human errors. >>>> With C5, we are trying to minimize the configurations, command >>>> line parameters, etc. to make the users' life easier and to reduce the >>>> human errors much as possible. Log patch is just a one case I highlighted. >>>> There are other use cases like; >>>> >>>>- Adding DB drivers >>>>- Configuring additional transports e.g: JMS >>>>- Putting custom mediators (in C4), etc. >>>> >>>> Above requirements are based on the C4 experience and there will >>>> definitely be some similar set of requirements in the C5 as well. Hence, if >>>> we introduce a separate command line, the user will have to start the >>>> server with that parameter each time they modify the lib directory, even >>>> they do a version upgrade of a given library. >>>> Hence I would prefer to make the library update >>>> decision programmatically rather a user input. >>>> >>>> @Lackman: >>>> With C5, we will abandon the word "patch". Everything including >>>> features, fixes, etc. will be provided via updates. Hence there is no point >>>> having a separate directory call "patches". A "log patch" is just another >>>> jar file having the same bundle symbolic name and version as its original >>>> bundle in the plugins directory. Once it is removed from the lib directory, >>>> the server will use the original bundle in the plugins directory. >>>> >>>> Thanks, >>>> Jayanga. >>>> >>>> >>>> >>>> *Jayanga Dissanayake* >>>> Associate Technical Lead >>>> WSO2 Inc. - http://wso2.com/ >>>> lean . enterprise . middleware >>>> email: jaya...@wso2.com >>>> mobile: +94772207259 <+94%2077%20220%207259> >>>> <http://wso2.com/signature> >>>> >>>> On Thu, May 18, 2017 at 12:06 PM, Lakshman Udayakantha < >>>> lakshm...@wso2.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> I also think running a separate tool or add startup parameter to say >>>>> there is a change to the existing bundles and deploy them also is a bette
Re: [Architecture] [APIM][C5] Removing "Blocked" state from API lifecycle
ked it can be done using 2,3 or 4. >>>> >>>> Please share your thoughts on this. >>>> >>>> Regards, >>>> Yasima. >>>> >>>> -- >>>> http://wso2.com/signatureYasima Dewmini >>>> Software Engineer, WSO2, Inc. >>>> Email: yas...@wso2.com >>>> Mobile: +94713117081 <+94%2071%20311%207081> >>>> >>> >>> >>> >>> -- >>> Nuwan Dias >>> >>> Software Architect - WSO2, Inc. http://wso2.com >>> email : nuw...@wso2.com >>> Phone : +94 777 775 729 <077%20777%205729> >>> >> >> >> >> -- >> >> *Sanjeewa Malalgoda* >> WSO2 Inc. >> Mobile : +94713068779 <+94%2071%20306%208779> >> >> <http://sanjeewamalalgoda.blogspot.com/>blog >> :http://sanjeewamalalgoda.blogspot.com/ >> <http://sanjeewamalalgoda.blogspot.com/> >> >> >> > > > -- > Nuwan Dias > > Software Architect - WSO2, Inc. http://wso2.com > email : nuw...@wso2.com > Phone : +94 777 775 729 <+94%2077%20777%205729> > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0717429601* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] C5 Modifying OSGi bundle deployment logic to support temporary patches (log patches, etc)
"plugins" directory and adds the external bundles on top of >>>> it. >>>> >>>> Recently we were looking into the C5 update/patching model and there >>>> was a concern on how we could add a temporary log patches, etc. to identify >>>> an issue. (in a case where all other possibilities failed). >>>> >>>> The most convenient to the client is to just add the given log patch to >>>> the pack, restart, collect the relevant logs, and finally revert the log >>>> patch. Any additional steps would be an overhead as the client is already >>>> undergoing a severe issue and client is trying to help us to identify the >>>> issue by applying the log patch. Hence this should be modeled in a >>>> convenient way to the users. >>>> >>>> There is a possibility to let the clients add the log patch into the >>>> [product_home]/lib directory and remove it once done. But with the current >>>> OSGi bundle deployment logic, we can't do that as it always gives the >>>> priority to the bundles in the plugins directory. If we put a bundle with >>>> the same bundle-symbolic-name and version into the plugins directory it >>>> will be ignored. >>>> >>>> To achieve this behavior we have to modify the existing OSGi bundle >>>> deployment logic as follows. >>>> >>>> 1. In the first run make a backup of the original bundles.info file >>>> (bundles.info.original this will be used as the baseline for each time it >>>> updated the bundles.info file) >>>> 2. Read the bundles.info.original file >>>> 3. Add the bundles in the [product_home]/lib directory >>>> 4. Update the bundles.info file >>>> And >>>> The logic in selecting effective bundles list should be changed to not >>>> to give priority to bundles in the plugins directory. Instead modify the >>>> entries, if a similar bundle (symbolic name and version) is found in the >>>> [product_home]/lib >>>> >>>> Above suggested approach allows easily add the patched jars into the >>>> [product_home]/lib directory for temporary purposes. And reverting the >>>> patch is also possible as we have a backup of the original bundles.info >>>> file >>>> >>>> WDYT? >>>> >>>> [1] https://github.com/wso2/carbon-kernel/blob/v5.2.0-m3/lau >>>> ncher/src/main/java/org/wso2/carbon/launcher/extensions/OSGi >>>> LibBundleDeployerUtils.java >>>> >>>> Thanks, >>>> *Jayanga Dissanayake* >>>> Associate Technical Lead >>>> WSO2 Inc. - http://wso2.com/ >>>> lean . enterprise . middleware >>>> email: jaya...@wso2.com >>>> mobile: +94772207259 <+94%2077%20220%207259> >>>> <http://wso2.com/signature> >>>> >>> >>> >>> >>> -- >>> >>> *Ruwan Abeykoon* >>> *Associate Director/Architect**,* >>> *WSO2, Inc. http://wso2.com <https://wso2.com/signature> * >>> *lean.enterprise.middleware.* >>> >>> >> >> ___ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Best Regards, > > *Vidura Nanayakkara* > Software Engineer > > Email : vidu...@wso2.com > Mobile : +94 (0) 717 919277 <+94%2071%20791%209277> > Web : http://wso2.com > Blog : https://medium.com/@viduran > LinkedIn : https://lk.linkedin.com/in/vidura-nanayakkara > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0717429601* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [C5] Carbon Secure Vault YAML Configuration
Adding these files to parameter section makes no difference because
parameter section includes implementation specific data. So I think it is
ok to keep them in parameter section as well. It is up to the
implementation.
On Fri, Mar 17, 2017 at 10:42 AM, Vidura Nanayakkara <vidu...@wso2.com>
wrote:
> Hi Kishanthan,
>
> When we have the master-keys.yaml and secrets.properties yaml file path in
> secure-vault.yaml configuration the end user do not need to override a
> method to explain how the paths are taken but rather handled within the
> secure vauilt implementation itself. So if the paths are specified as a
> relative path, we can locate the files using the relevant configuration
> file location (OSGi / non-OSGi) and if we specify the absolute path, we can
> locate the file straight away from the specified location.
>
>
> On Fri, Mar 17, 2017 at 10:28 AM, Vidura Nanayakkara <vidu...@wso2.com>
> wrote:
>
>> Hi Niranjan,
>>
>> Ideally, the OSGi / non-OSGi check should happen at the secure vault
>> initialization phase. The rest of the execution should happen accordingly
>> without checking for OSGi / non-OSGi. However, if we delegate providing
>> other file paths (secret.properties, master-key.yaml) to relevant
>> implementation classes we need to get the file location accordingly (OSGi /
>> non-OSGi). This will require an OSGi / non-OSGi check (even if we set the
>> OSGi / non-OSGi config path during initialization how would the end
>> developer know when overriding a specific method?)
>>
>> Example:
>>
>> @Override
>> public Path getMasterKeyYAMLPath() throws SecureVaultException {
>> Path masterKeysFilePath;
>> if (SecureVaultUtils.isOSGIEnv()) {
>> Path carbonHomePath = Utils.getCarbonHome();
>> masterKeysFilePath = Paths.get(carbonHomePath.toString(),
>> MASTER_KEYS_FILE_NAME);
>> } else {
>> Optional resourcePath = SecureVaultUtils
>> .getResourcePath("securevault", "conf",
>> MASTER_KEYS_FILE_NAME);
>> if (!resourcePath.isPresent()) {
>> throw new SecureVaultException(MASTER_KEYS_FILE_NAME +
>> "not found");
>> }
>> masterKeysFilePath = resourcePath.get();
>> }
>> return masterKeysFilePath;
>> }
>>
>>
>> We do not need to do an OSGi / non-OSGi check if we include the
>> secrets.properties and master-keys.yaml in the secure-vault.yaml as stated
>> above since we can keep the relevant paths in memory accordingly during the
>> secure vault initialization phase. Furthermore, is it really required to
>> delegate providing other file paths to the relevant implementation when we
>> can handle this from the secure-vault.yaml itself?
>>
>> My suggestion is to specify the locations in the secure-vault.yaml. This
>> way if we specify a relative path, we can get the path according to the
>> OSGi / non-OSGi mode (predefined configuration locations). If we specify an
>> absolute path we can take the path as it is to locate the files.
>>
>> WDYT?
>>
>> On Fri, Mar 17, 2017 at 10:18 AM, Niranjan Karunanandham <
>> niran...@wso2.com> wrote:
>>
>>> Hi Jayanga,
>>>
>>>
>>> On Fri, Mar 17, 2017 at 10:09 AM, Jayanga Dissanayake <jaya...@wso2.com>
>>> wrote:
>>>
>>>> Hi Niranjan,
>>>>
>>>> You are correct we should follow the same way as msf4j to detect
>>>> whether it is OSGi mode or not.
>>>> The properties suggested are to avoid the OSGi mode check in several
>>>> places. With the suggested properties, secure-vault.yaml will have all the
>>>> information it needs for the initialization. Hence it could check the mode
>>>> at one place and initialize the secure vault accordingly.
>>>>
>>> Is it possible to move the parts where we need to check if it is an OSGi
>>> mode or not to a generic place that is at the start and these values are
>>> based to the implementation later? So that the implementation layer does
>>> not need to know whether it is a OSGi or non-OSGi.
>>>
>>>
>>>>
>>>> Thanks,
>>>> Jayanga.
>>>>
>>>> *Jayanga Dissanayake*
>>>> Associate Technical Lead
>>>> WSO2 Inc. - http://wso2.com/
>>>> lean . enterprise . middleware
>>>> email: jaya...@wso2.com
>>>> mobile: +94772207259 <+94%2077%20220%207259>
>>>
Re: [Architecture] [C5] Carbon Secure Vault YAML Configuration
Hi Vidura, On Fri, Mar 17, 2017 at 9:15 AM, Vidura Nanayakkara <vidu...@wso2.com> wrote: > Hi All, > > An example for a secure vault YAML configuration file is as shown below > according to the current implementation. > > secretRepository: > type: org.wso2.carbon.kernel.securevault.repository. > DefaultSecretRepository > parameters: > privateKeyAlias: wso2carbon > keystoreLocation: resources/security/wso2carbon.jks > masterKeyReader: > type: org.wso2.carbon.kernel.securevault.reader.DefaultMasterKeyReader > > However, according to the discussion made in [1] > <http://wso2-oxygen-tank.10903.n7.nabble.com/C5-Moving-Carbon-Configuration-and-Carbon-Sec-Vault-to-2-Separate-Repositories-Removing-from-Kernel-td146953.html> > , we decided to move Carbon Secure Vault out of Carbon Kernel for the > specified reasons in [1] > <http://wso2-oxygen-tank.10903.n7.nabble.com/C5-Moving-Carbon-Configuration-and-Carbon-Sec-Vault-to-2-Separate-Repositories-Removing-from-Kernel-td146953.html>. > According to this change, in OSGi mode the Secret repository and the > master key reader will be an implementation of the specified classes ( > org.wso2.carbon.kernel.securevault.repository.DefaultSecretRepository and > org.wso2.carbon.kernel.securevault.reader.DefaultMasterKeyReader) and > will be registered via the Secure Vault Component while in standalone > mode the secret repository and master key reader will be instances of the > specified classes and will be created using the class.forName() method. > > According to this implementation, it was decided to delegate providing > other file paths (secret.properties, master-key.yaml) to relevant > implementation classes because other file paths (secret.properties, > master-key.yaml) are bound to the relevant implementation. However, with > this approach, we are forced to check whether the code is being executed in > OSGi mode or non-OSGi mode in order to provide the correct location of the > file paths (secret.properties, master-key.yaml). > Since this happens in implementation class as in this case in Default implementation, IMO it is not a problem to check whether OSGI or not to give the correct file location. Even when you create another implementation that should work in both OSGI and non OSGI enviorenments you have to check for OSGI or not to give the correct file location. > > > *Suggestion:* > > secretRepository: > type: org.wso2.carbon.secvault.securevault.repository. > DefaultSecretRepository > parameters: > privateKeyAlias: wso2carbon > keystoreLocation: securevault/resources/security/wso2carbon.jks > secretProperties: securevault/resources/security/secrets.properties > masterKeyReader: > type: org.wso2.carbon.secvault.securevault.utils. > DefaultHardCodedMasterKeyReader > parameters: > masterKeyFile: securevault/resources/security/master-keys.yaml > > > If we could add the highlighted properties to the secure vault YAML > configuration file specifying the location of the master-keys.yaml and > secrets.properties, we only need to check whether the code is being > executed in OSGi mode or non-OSGi mode once at the time of secure vault > initialisation. > > WDYT? > > [1] [C5] Moving Carbon Configuration and Carbon Sec-Vault to 2 Separate > Repositories (Removing from Kernel) > <http://wso2-oxygen-tank.10903.n7.nabble.com/C5-Moving-Carbon-Configuration-and-Carbon-Sec-Vault-to-2-Separate-Repositories-Removing-from-Kernel-td146953.html> > > > Best Regards, > > *Vidura Nanayakkara* > Software Engineer > > Email : vidu...@wso2.com > Mobile : +94 (0) 717 919277 <+94%2071%20791%209277> > Web : http://wso2.com > Blog : https://medium.com/@viduran <http://wso2.com/> > Twitter : http://twitter.com/viduranana > LinkedIn : https://lk.linkedin.com/in/vidura-nanayakkara > <http://wso2.com/> > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0717429601* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [C5] Moving Carbon Configuration and Carbon Sec-Vault to 2 Separate Repositories (Removing from Kernel)
Hi Imesh, On Fri, Mar 10, 2017 at 3:54 PM, Imesh Gunaratne <im...@wso2.com> wrote: > Hi Vidura, > > I think it would be better if we can first move the secure vault code from > carbon-kernel repository to the new repository with commit history and then > apply the changes you have done. Otherwise, we will loose all history. > +1 to preserve history. > > I had a chat with Lakshman on this and it seems like he has extracted all > secure-vault related code into a new component and sent a PR [1] but it has > not been merged. > > IMO we would need to do following: > >- First, fix conflicts and merge [1]. This would bring all secure >vault related code to a new component/folder. > > I have solved the conflicts and updated the PR. Thanks, Lakshman. > >- >- Then move above folder to [2] using a PR-X >- Once the PR-X is merged, apply your changes on top of it. > > [1] https://github.com/wso2/carbon-kernel/pull/1266 > [2] https://github.com/wso2/carbon-secvault > > Thanks > > On Mon, Mar 6, 2017 at 12:15 PM, Niranjan Karunanandham <niran...@wso2.com > > wrote: > >> Hi Vidura, >> >> On Mon, Mar 6, 2017 at 11:52 AM, Imesh Gunaratne <im...@wso2.com> wrote: >> >>> On Fri, Mar 3, 2017 at 12:00 PM, Thusitha Thilina Dayaratne < >>> thusit...@wso2.com> wrote: >>> >>>> Rather than having a separate repo for utils I'll look into the >>>> possibility of moving that to a separate component (same level as core) >>>> without having cyclic dependencies. If that is possible then we can pack >>>> that as a new feature or core feature itself. Otherwise lets move that to a >>>> separate repo. >>>> >>>> Thusitha has done this change in the following PR: >>> https://github.com/wso2/carbon-kernel/pull/1318 >>> >> Once this PR is merged, we need to add the support to provide an API >> which can return the current config folder for a particular runtime. >> >> >>> >>> >>> Thanks >>> >>> -- >>> *Imesh Gunaratne* >>> Software Architect >>> WSO2 Inc: http://wso2.com >>> T: +94 11 214 5345 M: +94 77 374 2057 <+94%2077%20374%202057> >>> W: https://medium.com/@imesh TW: @imesh >>> lean. enterprise. middleware >>> >>> >> Regards, >> Nira >> >> -- >> >> >> *Niranjan Karunanandham* >> Associate Technical Lead - WSO2 Inc. >> WSO2 Inc.: http://www.wso2.com >> >> > > > -- > *Imesh Gunaratne* > Software Architect > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 <+94%2077%20374%202057> > W: https://medium.com/@imesh TW: @imesh > lean. enterprise. middleware > > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0717429601* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [ESB] Persistent TCP connection in Axis2 Transport
Hi IsuruH,
Sorry for the delayed response. Find inline comments.
On Wed, Feb 15, 2017 at 7:43 AM, Isuru Haththotuwa <isu...@wso2.com> wrote:
> Hi Lakshman,
>
> On Thu, Jan 26, 2017 at 10:57 PM, Lakshman Udayakantha <lakshm...@wso2.com
> > wrote:
>
>> Hi,
>>
>> *Usecases*
>>
>> Sending messages To TCP backend via a persistent connection in ESB. ESB
>> will use TCPTransportSender class to send TCP messages when send mediator
>> used with TCP protocol.
>> This messages will send after adding message length to the front of the
>> message. Back end also should send messages by adding message length to the
>> front of the messages for the purpose of reading easily.
>> Messages can be concatenated in TCP layer even though application layer
>> send messages separately. There can be delay in message reading from TCP
>> axi2 end and that delay will lead to concatenation of messages. Even though
>> messages concatenated message should be retrieved separately and send
>> separately to the original caller.
>> ESB should recognized disconnection of TCP links immediately and retry in
>> configured time interval for configured number of times and if failed even
>> after, should report to the client about the error.
>>
>> *Solution for persistent connection.*
>>
>> TCPTransportSender {
>>
>> private Map<String, Socket> persistentConnectionsMap = new HashMap<>();
>>
>> }
>>
>> This map holds a unique client id and a socket instance. Whenever a
>> message comes to TCPTransportSender it retrieves the relevant socket
>> instance to the relevant client id and will send messages via that socket.
>>
>> *How to prevent message concatenation in TCP layer.*
>>
>> Existing methods flow is as follows for messages
>>
>> sendMessage() --> writeMessageOut() --> waitForReply() --> getMessage()
>>
>> According to the current implementation getMessage() will receive
>> messages from TCP backend and after receiving messages it will read bytes
>> for message length which is in front of the message ,via a buffer. After
>> that it will read the actual message by message length via the same buffer.
>> Since getMessage will return after reading the first message, it will lost
>> the second message or snippet from it which is remaining in the buffer.
>> Then what kind of mechanism should we implement to process the subsequent
>> messages also and send them separately to the original caller (websocket
>> inbound, tcp client, etc )?
>>
> I'm not sure if you have found a solution and implemented this already.
> Can we use a specific delimiting character to indicate that the message
> sequence is complete? It should read the byte buffer each time, with the
> length specified, and if it encounters this delimiter, should assume that
> there is no more to read for that particular message sequence. WDYT? If you
> have a better solution implemented already, please do share.
>
According to the requirement, we need to prefixed message length to message
itself. So no point of adding any other delimiting character to indicate
the end of the message. End of the message will identify by the message
length bytes that can be found in front of the message.
For the purpose of detecting TCP connection immediately, It was created
another thread which is indefinitly wait for incoming messages. TCP layer
will send -1 if any diconnection happen. So we can use that to throw an
error to the original caller.
Possible message concatenation happen in TCP layer is avoided by adding a
Map that holds response message contexts with client IDs. So even though
concatenated message arrived, relevant response message context will be
retrieved from Map and will process separately.
According to the current implementation done, For the persistent
connection, three threads will be created for message processing.
1. TCPTransportSender - Responsible for message sending to the backend.
2. TCPBackendListener - Responsilbe for waiting for messages from backend
indefinitely. This thread will share a blocking queue which holds
exceptions, with TCPTransportSender thread. If the connection broken, an
exception will add to the the exception blocking queue. Then
TCPTransportSender thread will use retry mechanism mentioned in first mail
if specified, Or will issue the exception to the original caller.
3. TCPResponseSender - Responsible for message sending to the original
caller. This thread will share a blocking queue which holds incoming
messages, with TCPBackendListener thread and another blocking queue which
holds response message contexts, with TCPTransportSender thread. When a
message arrived, relevant message and response messag
Re: [Architecture] [ESB] Persistent TCP connection in Axis2 Transport
Hi Jagath,
Thanks for the suggestions provided. We will apply them when implementing.
Thanks.
On Sat, Jan 28, 2017 at 12:40 AM, Jagath Sisirakumara Ariyarathne <
jaga...@wso2.com> wrote:
> Hi Lakshman,
>
> When handling multiple responses using a single persistent connection, we
> need to use some kind of unique key per request and send it to the back-end
> with the request. When the back-end responses, it should include the above
> key in the response as well. It will help to map the request and response.
> Also it will be useful when multiple responses are available for a single
> request.
>
> According to the existing implementation, it creates a new socket per
> request, and closes it once the response is received. With this new
> implementation, it should not close the connection and should keep reading
> other responses as well. IMO here we can keep connections per host:port
> (destination) instead of per client. It will reduce the number of
> persistent connections established with back-end.
>
> With this requirement,communication with the back-end needs to be handled
> in a separate thread because it is required to continuously wait for data
> from the same connection.
>
> Thanks.
>
>
> On Fri, Jan 27, 2017 at 10:53 AM, Lakshman Udayakantha <lakshm...@wso2.com
> > wrote:
>
>> Hi Chanaka,
>>
>> Thanks for your suggestions. As you said this was supposed to be
>> developed on top of TCP transport protocol. Actually, This implementation
>> will be done by assuming both ends should agree upon the way of messages
>> send. Anyhow messages concatenation haven't done by the TCP backend. It can
>> happen in the TCP layer even though application layer sent those message
>> separately because if some delay in ESB to read messages will lead to
>> flooded the messages in TCP layer and this will cause two messages come in
>> one lump to the ESB. Shouldn't we handle that case in ESB Axis2 transport
>> side?
>>
>> Thanks,
>> Lakshman
>>
>> On Fri, Jan 27, 2017 at 7:31 AM, Chanaka Fernando <chana...@wso2.com>
>> wrote:
>>
>>> Hi Lakshman,
>>>
>>> I think you are trying to implement a feature on top of the TCP
>>> transport protocol. At the TCP level, it handles the reliable message
>>> communication through its protocol specific features (e.g. sequence number,
>>> resend). If you want to implement something on top of that, both your
>>> client and the server (back end) should come to an agreement on how the
>>> messages are sent and received. For example, If the back end sends several
>>> messages concatenated as a single message, it must include that information
>>> on the message somehow, so that the receiving side can properly read that
>>> message. Otherwise, there is no other way to read that message correctly
>>> from the client side. For this solution, you need to design both client and
>>> back end side of the communication.
>>>
>>>
>>> On Thu, Jan 26, 2017 at 10:57 PM, Lakshman Udayakantha <
>>> lakshm...@wso2.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> *Usecases*
>>>>
>>>> Sending messages To TCP backend via a persistent connection in ESB. ESB
>>>> will use TCPTransportSender class to send TCP messages when send mediator
>>>> used with TCP protocol.
>>>> This messages will send after adding message length to the front of the
>>>> message. Back end also should send messages by adding message length to the
>>>> front of the messages for the purpose of reading easily.
>>>> Messages can be concatenated in TCP layer even though application layer
>>>> send messages separately. There can be delay in message reading from TCP
>>>> axi2 end and that delay will lead to concatenation of messages. Even though
>>>> messages concatenated message should be retrieved separately and send
>>>> separately to the original caller.
>>>> ESB should recognized disconnection of TCP links immediately and retry
>>>> in configured time interval for configured number of times and if failed
>>>> even after, should report to the client about the error.
>>>>
>>>> *Solution for persistent connection.*
>>>>
>>>> TCPTransportSender {
>>>>
>>>> private Map<String, Socket> persistentConnectionsMap = new HashMap<>();
>>>>
>>>> }
>>>>
>>>> This map holds a unique client id and a socket instance. Whenever a
>>>> message comes
Re: [Architecture] [ESB] Persistent TCP connection in Axis2 Transport
Hi Chanaka,
Thanks for your suggestions. As you said this was supposed to be developed
on top of TCP transport protocol. Actually, This implementation will be
done by assuming both ends should agree upon the way of messages send.
Anyhow messages concatenation haven't done by the TCP backend. It can
happen in the TCP layer even though application layer sent those message
separately because if some delay in ESB to read messages will lead to
flooded the messages in TCP layer and this will cause two messages come in
one lump to the ESB. Shouldn't we handle that case in ESB Axis2 transport
side?
Thanks,
Lakshman
On Fri, Jan 27, 2017 at 7:31 AM, Chanaka Fernando <chana...@wso2.com> wrote:
> Hi Lakshman,
>
> I think you are trying to implement a feature on top of the TCP transport
> protocol. At the TCP level, it handles the reliable message communication
> through its protocol specific features (e.g. sequence number, resend). If
> you want to implement something on top of that, both your client and the
> server (back end) should come to an agreement on how the messages are sent
> and received. For example, If the back end sends several messages
> concatenated as a single message, it must include that information on the
> message somehow, so that the receiving side can properly read that message.
> Otherwise, there is no other way to read that message correctly from the
> client side. For this solution, you need to design both client and back end
> side of the communication.
>
>
> On Thu, Jan 26, 2017 at 10:57 PM, Lakshman Udayakantha <lakshm...@wso2.com
> > wrote:
>
>> Hi,
>>
>> *Usecases*
>>
>> Sending messages To TCP backend via a persistent connection in ESB. ESB
>> will use TCPTransportSender class to send TCP messages when send mediator
>> used with TCP protocol.
>> This messages will send after adding message length to the front of the
>> message. Back end also should send messages by adding message length to the
>> front of the messages for the purpose of reading easily.
>> Messages can be concatenated in TCP layer even though application layer
>> send messages separately. There can be delay in message reading from TCP
>> axi2 end and that delay will lead to concatenation of messages. Even though
>> messages concatenated message should be retrieved separately and send
>> separately to the original caller.
>> ESB should recognized disconnection of TCP links immediately and retry in
>> configured time interval for configured number of times and if failed even
>> after, should report to the client about the error.
>>
>> *Solution for persistent connection.*
>>
>> TCPTransportSender {
>>
>> private Map<String, Socket> persistentConnectionsMap = new HashMap<>();
>>
>> }
>>
>> This map holds a unique client id and a socket instance. Whenever a
>> message comes to TCPTransportSender it retrieves the relevant socket
>> instance to the relevant client id and will send messages via that socket.
>>
>> *How to prevent message concatenation in TCP layer.*
>>
>> Existing methods flow is as follows for messages
>>
>> sendMessage() --> writeMessageOut() --> waitForReply() --> getMessage()
>>
>> According to the current implementation getMessage() will receive
>> messages from TCP backend and after receiving messages it will read bytes
>> for message length which is in front of the message ,via a buffer. After
>> that it will read the actual message by message length via the same buffer.
>> Since getMessage will return after reading the first message, it will lost
>> the second message or snippet from it which is remaining in the buffer.
>> Then what kind of mechanism should we implement to process the subsequent
>> messages also and send them separately to the original caller (websocket
>> inbound, tcp client, etc )?
>>
>> *Solution to retry mechanism.*
>>
>> To provide two separate configurations,
>> retryInterval - in seconds - default 3 seconds
>> noOfRetries - default 3
>>
>> and finally send an exception report to the client.
>>
>> Your suggestions are highly appreciated.
>>
>> Thanks.
>> --
>> Lakshman Udayakantha
>> WSO2 Inc. www.wso2.com
>> lean.enterprise.middleware
>> Mobile: *0717429601 <071%20742%209601>*
>>
>>
>
>
> --
> Thank you and Best Regards,
> Chanaka Fernando
> Senior Technical Lead
> m: +94 773337238 <+94%2077%20333%207238>
> https://wso2.com <https://wso2.com/signature>
>
>
>
>
>
>
>
--
Lakshman Udayakantha
WSO2 Inc. www.wso2.com
lean.enterprise.middleware
Mobile: *0717429601*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] [ESB] Persistent TCP connection in Axis2 Transport
Hi,
*Usecases*
Sending messages To TCP backend via a persistent connection in ESB. ESB
will use TCPTransportSender class to send TCP messages when send mediator
used with TCP protocol.
This messages will send after adding message length to the front of the
message. Back end also should send messages by adding message length to the
front of the messages for the purpose of reading easily.
Messages can be concatenated in TCP layer even though application layer
send messages separately. There can be delay in message reading from TCP
axi2 end and that delay will lead to concatenation of messages. Even though
messages concatenated message should be retrieved separately and send
separately to the original caller.
ESB should recognized disconnection of TCP links immediately and retry in
configured time interval for configured number of times and if failed even
after, should report to the client about the error.
*Solution for persistent connection.*
TCPTransportSender {
private Map<String, Socket> persistentConnectionsMap = new HashMap<>();
}
This map holds a unique client id and a socket instance. Whenever a message
comes to TCPTransportSender it retrieves the relevant socket instance to
the relevant client id and will send messages via that socket.
*How to prevent message concatenation in TCP layer.*
Existing methods flow is as follows for messages
sendMessage() --> writeMessageOut() --> waitForReply() --> getMessage()
According to the current implementation getMessage() will receive messages
from TCP backend and after receiving messages it will read bytes for
message length which is in front of the message ,via a buffer. After that
it will read the actual message by message length via the same buffer.
Since getMessage will return after reading the first message, it will lost
the second message or snippet from it which is remaining in the buffer.
Then what kind of mechanism should we implement to process the subsequent
messages also and send them separately to the original caller (websocket
inbound, tcp client, etc )?
*Solution to retry mechanism.*
To provide two separate configurations,
retryInterval - in seconds - default 3 seconds
noOfRetries - default 3
and finally send an exception report to the client.
Your suggestions are highly appreciated.
Thanks.
--
Lakshman Udayakantha
WSO2 Inc. www.wso2.com
lean.enterprise.middleware
Mobile: *0717429601*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [APIM][C5] Workflow Implementation
Hi, Earlier we used BPS as the default workflow engine. What is the default workflow engine we are going to use with C5 implementation? And There should be a pluggable point to plug any workflow engine in there. Thanks, Lakshman On Thu, Jan 26, 2017 at 12:08 PM, Shani Ranasinghe <sh...@wso2.com> wrote: > Hi Yasima, > > Just one thing, IMO we need to incorporate the complete process if the > workflow is enabled also right? in that case the workflow engine will call > APIM with the status. > > On Thu, Jan 26, 2017 at 11:29 AM, Yasima Dewmini <yas...@wso2.com> wrote: > >> Hi all, >> >> We are going to implement workflow extensions for APIM. >> Following diagram shows the proposed solution. >> >> Your thoughts and suggestions regarding this are highly appreciated. >> >> Regards, >> Yasima. >> >> -- >> http://wso2.com/signatureYasima Dewmini >> Software Engineer, WSO2, Inc. >> Email: yas...@wso2.com >> Mobile: +94713117081 <+94%2071%20311%207081> >> >> ___ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Thanks and Regards > *,Shani Ranasinghe* > Senior Software Engineer > WSO2 Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: +94 77 2273555 <+94%2077%20227%203555> > Blog: http://waysandmeans.blogspot.com/ > linked in: lk.linkedin.com/pub/shani-ranasinghe/34/111/ab > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0717429601* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Non OSGI access to secure vault component
Hi,
There are two functionalities can be identified in the secure vault
loading.
1. Reading the secure-vault.yaml and initializing instance variables from
values in file.
2. Initialising the master key reader secret repository and load secrets to
secret repository.
According to the desgin these functionalities reside in
SecureVaultComponent. After refactoring the code reading secure-vault.yaml
and initialising and common code moved to a new class called
SecureVaultInitializer. Code related to OSGI has been kept in
SecureVaultComponent class and common functions have moved to
SecureVaultInitializer class. After refactoring basic class and method
structures as follows.
Class SecureVaultInitializer {
public void initFromSecureVaultYAML() {
//reading secure vault yaml and initialising runtime variables happens here
}
public SecureVault initializeSecureVault(String masterKeyFilePath, String
secretPropertiesFilePath, String secureVaultYAMLPath) {
//initialise secure vault for SPI case
}
public SecureVault initializeSecureVault() {
//initialise the securevault in general.
}
}
Class SecureVaultComponent {
public SecureVaultComponent() {
//call SecureVaultInitilizer.initFromSecureVaultYAML method to read
secure vault yaml and initialise runtime variables
}
private void initializeSecureVault() {
// waiting for the references to resolve, call
SecureVaultInitializer.initializeSecureVault general case and register
securevault service in OSGI service registry
}
}
Class NonOSGIInvoker {
public static void main ( String[] args )
{
SecureVaultInitializer.getInstance().initializeSecureVault(masterKeysFilePath,
secretPropertiesFilePath, secureVaultYAMLPath).resolve(alias);
}
}
After discussion with team, It was decided to make entries in secure-vault
yaml optional and to provide one implementation of the providers of secret
repository and master key readers in class path. But this seems impossible
because there is already one implementation in securevault bundle which is
the default implementation. If non osgi secure vault coded to read only one
implementation, it always pick the default implementation because it first
look inside the bundle. Please raise any concern with the current
implementation to make it better.
Thanks,
Lakshman.
On Wed, Jan 4, 2017 at 10:23 PM, Lakshman Udayakantha <lakshm...@wso2.com>
wrote:
> Hi,
>
> The current implementation of secure vault functionality reside in carbon
> core component. So to make secure vault functionality available for non
> OSGI context, It can be separated out this piece of functionality as an
> OSGI bundle and remove tight coupling in code with OSGI context and users
> who wants to access non OSGI secure vault can get the separated bundle as a
> maven dependency. According to the current design there are three main sub
> components in secure vault.
>
> 1. Secret Repository
> 2. Master Key Reader
> 3. Secure Vault OSGI service.
>
> It can be plugged any secret repository implementation or any master key
> reader implementation to the code with the current implementation as
> pointed out in diagram [1].
> To expose these functionality via java SPI, it can be loaded secret
> repository and master key reader implementations to SecureVaultComponent
> via ServiceLoader class. Custom Secret Repositories and Custom Master key
> readers will be service providers. Then current SecureVault interface and
> implementation can be used without any redesign in code. Appreciate your
> thoughts and concerns on this.
>
> [1] SecureVault UML
> <https://drive.google.com/file/d/0B9KDy4GJKr1vS0NUOG5CRXhpMDg/view?usp=sharing>
>
> Thanks,
> --
> Lakshman Udayakantha
> WSO2 Inc. www.wso2.com
> lean.enterprise.middleware
> Mobile: 0717429601
>
--
Lakshman Udayakantha
WSO2 Inc. www.wso2.com
lean.enterprise.middleware
Mobile: *0717429601*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] WebSocket support for MSF4J
e. If we do so MSF4J might get slower since >>>>> it >>>>> needs to check whether every incoming message is a WebSocket message or a >>>>> HTTP message. >>>>> Also for Server-Push we have to add another interface to >>>>> Carbon-Messaging >>>>> So what would be the better approach? >>>>> >>>>> Thanks, >>>>> Irunika >>>>> >>>>> *Irunika Weeraratne* >>>>> *Software Engineer | WSO2, Inc. <http://wso2.com/>* >>>>> *Email : irun...@wso2.com <irun...@wso2.com>* >>>>> *LinkedIn : https://lk.linkedin.com/in/irunika >>>>> <https://lk.linkedin.com/in/irunika>* >>>>> *Mobile : +94712403314 <+94%2071%20240%203314>* >>>>> *Lean . Enterprise . Middleware* >>>>> >>>>> >>>>> On Fri, Dec 9, 2016 at 12:07 PM, Irunika Weeraratne <irun...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> After the offline discussion with Azeez and Kishanthan, we identified >>>>>> 2 main issues with the current approach >>>>>> >>>>>>1. Current design compromises the very reason of putting the >>>>>>Carbon-Messaging in between Carbon-Transport and MSF4J which is >>>>>> decoupling >>>>>>the Transport layer from MSF4J since it has a direct dependency to >>>>>>Carbon-Transport from MSF4J. >>>>>>So now we intend to add new interfaces and extended >>>>>>CarbonMessages to Carbon-Messaging which are compatible with WebSocket >>>>>>protocol. >>>>>>2. We have to think about the Load Balancer awareness of a given >>>>>>WebSocket connection since the WebSocket connection should be >>>>>> persistent >>>>>>until the client or the server closes it. So we have to identify how >>>>>> we can >>>>>>persist the connection in server side with multiple nodes. >>>>>> >>>>>> Will come up with a design and update. >>>>>> If you have any thoughts please share. >>>>>> >>>>>> *Irunika Weeraratne* >>>>>> *Software Engineer | WSO2, Inc. <http://wso2.com/>* >>>>>> *Email : irun...@wso2.com <irun...@wso2.com>* >>>>>> *LinkedIn : https://lk.linkedin.com/in/irunika >>>>>> <https://lk.linkedin.com/in/irunika>* >>>>>> *Mobile : +94712403314 <+94%2071%20240%203314>* >>>>>> *Lean . Enterprise . Middleware* >>>>>> >>>>>> >>>>>> On Wed, Dec 7, 2016 at 3:31 PM, Irunika Weeraratne <irun...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> I'm currently working on the implementation of WebSocket support for >>>>>>> MSF4J. >>>>>>> >>>>>>> *Overview* >>>>>>> >>>>>>> >>>>>>> Currently, Carbon-Transport only supports HTTP. So we need to give >>>>>>> it the ability to handle WebSocket Frames. >>>>>>> Also, we are planning to use existing Carbon-Messaging to process >>>>>>> incoming messages. But in WebSocket, a response for a request is not a >>>>>>> must. User has to handle it if there is a need of sending a response. >>>>>>> Also, we need to give the user the ability to do server pushes using >>>>>>> the WebSocket Sessions and to do so we need to have the reference of >>>>>>> Netty's Channel Handler Context. >>>>>>> >>>>>>> In MSF4J >>>>>>> >>>>>>>- Need a separate Registry to register WebSocket EndPoints since >>>>>>>WebSoocket EndPoints are different from MicroServices >>>>>>>- Message Processor gets the correct EndPoint from the registry >>>>>>>and dispatch the message >>>>>>> >>>>>>> >>>>>>> Any thoughts? >>>>>>> >>>>>>> Thanks & Best Regards, >>>>>>> Irunika >>>>>>> *Irunika Weeraratne* >>>>>>> *Software Engineer | WSO2, Inc. <http://wso2.com/>* >>>>>>> *Email : irun...@wso2.com <irun...@wso2.com>* >>>>>>> *LinkedIn : https://lk.linkedin.com/in/irunika >>>>>>> <https://lk.linkedin.com/in/irunika>* >>>>>>> *Mobile : +94712403314 <+94%2071%20240%203314>* >>>>>>> *Lean . Enterprise . Middleware* >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> >>> -- >>> *Kishanthan Thangarajah* >>> Technical Lead, >>> Platform Technologies Team, >>> WSO2, Inc. >>> lean.enterprise.middleware >>> >>> Mobile - +94773426635 <+94%2077%20342%206635> >>> Blog - *http://kishanthan.wordpress.com >>> <http://kishanthan.wordpress.com>* >>> Twitter - *http://twitter.com/kishanthan >>> <http://twitter.com/kishanthan>* >>> >> >> > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0717429601* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Non OSGI access to secure vault component
Hi, The current implementation of secure vault functionality reside in carbon core component. So to make secure vault functionality available for non OSGI context, It can be separated out this piece of functionality as an OSGI bundle and remove tight coupling in code with OSGI context and users who wants to access non OSGI secure vault can get the separated bundle as a maven dependency. According to the current design there are three main sub components in secure vault. 1. Secret Repository 2. Master Key Reader 3. Secure Vault OSGI service. It can be plugged any secret repository implementation or any master key reader implementation to the code with the current implementation as pointed out in diagram [1]. To expose these functionality via java SPI, it can be loaded secret repository and master key reader implementations to SecureVaultComponent via ServiceLoader class. Custom Secret Repositories and Custom Master key readers will be service providers. Then current SecureVault interface and implementation can be used without any redesign in code. Appreciate your thoughts and concerns on this. [1] SecureVault UML <https://drive.google.com/file/d/0B9KDy4GJKr1vS0NUOG5CRXhpMDg/view?usp=sharing> Thanks, -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: 0717429601 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [EMM] Enabling application white listing and application blacklisting support
Hi Chathura, The approach you mentioned can be broken for two incidents mentioned below. 1. Remove a role from a user 2. Remove a role from an application If I explained first case, Let's suppose *application1* has restricted to *role1* and *user1* has *role1*. User will be able to see the *application1* and subscribed to it. After subscribed to *application1*, *role1* has removed from *user1*. After that also user can see *application1* is a subscribed application. If we get subscribed application list in this moment in order to send to the device for white listing, It should list *application1* (which is wrong) also. So shall we go with the approach that I mentioned or is there any better of doing this? Thanks On Tue, Mar 1, 2016 at 7:28 PM, Chathura Dilan <chathu...@wso2.com> wrote: > Hi Lakshman, > > > +1 for two options in EMM > > 1. White list is enabled or not > 2. Which app store is used to provide the white list > > > But the operation I think it is very expensive. > 1. Getting app list from the device ( If user has multiple devices, you > need to get them separately) > 2. Sending all of them to app manager to check the compliance. > 3. Then block apps which do not comply with each devices. > > Following operation would be much easy. > 1. Getting subscribed app list from the user in app manager. > 2. Sending that app list to user's devices for white listing. > > > > On Tue, Mar 1, 2016 at 5:32 AM, Lakshman Udayakantha <lakshm...@wso2.com> > wrote: > >> Hi Chathura et al, >> >> @Chathura:Thanks for the detailed information. >> >> As per the offline discussion with PrabathA, EMM should specify >> explicitly that EMM is using app manager white list. Therefore when policy >> is created, below information should provided. >> >> 1. White list is enabled or not >> 2. Which app store is used to provide the white list >> >> When consider policy compliance, It will get installed application list >> from device and will pass that application list to app manager. There >> should be an API in app manager to return true if application list has >> access to the user and if not return false with application list which is >> not access by user and if there is mismatch emm will pass with relevant >> application list to device to block them or uninstall. >> >> Please comment if you have any concern about this approach. >> >> Thanks >> >> On Mon, Feb 29, 2016 at 5:47 PM, Lakshman Udayakantha <lakshm...@wso2.com >> > wrote: >> >>> Hi All, >>> >>> After bit of discussion with EMM team, it is decided to move creation of >>> white list policy to the app manager it self. We will provide a >>> configuration whether white list is enabled or not. If white list enabled >>> in the configuration, a new policy will created in EMM when new application >>> is going to publish. All other later application will be added to existing >>> white list policy when they are going to publish. This new policy will list >>> in policy list and can be edit later to add a new policy criteria. >>> >>> When considering policy compliance, EMM will request application list >>> with roles and update the existing policy and will request installed >>> application list from device and will compare both for compliance Or >>> another option would be to update the existing policy when something happen >>> in app manager that would break existing app to role mapping. Ex: >>> >>> 1. Remove a role from a user >>> 2. Remove a role from an application >>> >>> and request installed application list from device and check both for >>> compliance. >>> >>> Another thing is these application restrictions will be implement based >>> on COPE at initial stage. BYOD scenario will be considered later because of >>> the complications mentioned in previous reply. >>> >>> WDYT about this approach? >>> >>> Thanks >>> >>> On Tue, Feb 23, 2016 at 3:26 PM, Lakshman Udayakantha < >>> lakshm...@wso2.com> wrote: >>> >>>> Hi DilanA/EMM Team, >>>> >>>> @DilanA :Thanks for the information. >>>> >>>> I have assumed policy creator know the package names of the >>>> applications which need to be restricted in the device and implemented the >>>> mdm policy UI for app restriction list and able to publish the restriction >>>> list to the device successfully as the first step. >>>> >>>> Some terminology has been changed after thi
Re: [Architecture] [EMM] Enabling application white listing and application blacklisting support
Hi Chathura et al, @Chathura:Thanks for the detailed information. As per the offline discussion with PrabathA, EMM should specify explicitly that EMM is using app manager white list. Therefore when policy is created, below information should provided. 1. White list is enabled or not 2. Which app store is used to provide the white list When consider policy compliance, It will get installed application list from device and will pass that application list to app manager. There should be an API in app manager to return true if application list has access to the user and if not return false with application list which is not access by user and if there is mismatch emm will pass with relevant application list to device to block them or uninstall. Please comment if you have any concern about this approach. Thanks On Mon, Feb 29, 2016 at 5:47 PM, Lakshman Udayakantha <lakshm...@wso2.com> wrote: > Hi All, > > After bit of discussion with EMM team, it is decided to move creation of > white list policy to the app manager it self. We will provide a > configuration whether white list is enabled or not. If white list enabled > in the configuration, a new policy will created in EMM when new application > is going to publish. All other later application will be added to existing > white list policy when they are going to publish. This new policy will list > in policy list and can be edit later to add a new policy criteria. > > When considering policy compliance, EMM will request application list with > roles and update the existing policy and will request installed application > list from device and will compare both for compliance Or another option > would be to update the existing policy when something happen in app manager > that would break existing app to role mapping. Ex: > > 1. Remove a role from a user > 2. Remove a role from an application > > and request installed application list from device and check both for > compliance. > > Another thing is these application restrictions will be implement based on > COPE at initial stage. BYOD scenario will be considered later because of > the complications mentioned in previous reply. > > WDYT about this approach? > > Thanks > > On Tue, Feb 23, 2016 at 3:26 PM, Lakshman Udayakantha <lakshm...@wso2.com> > wrote: > >> Hi DilanA/EMM Team, >> >> @DilanA :Thanks for the information. >> >> I have assumed policy creator know the package names of the applications >> which need to be restricted in the device and implemented the mdm policy UI >> for app restriction list and able to publish the restriction list to the >> device successfully as the first step. >> >> Some terminology has been changed after this thread initialised. As of >> now If AWL is enabled we will provide role based application access. Policy >> creator will define application white list with set of roles along with the >> application. Only those roles will be able to access the application. If >> ABL is enabled, policy creator will define black list via the UI and those >> application list will not be allowed to run on any device. >> >> @EMM Team: I got several questions regarding the restriction apps using >> mobile agent app. >> 1. If we provide AWL, only those applications will show in app manager >> store. Other app stores, side loading and google play store needs to be >> blocked. This kind of behaviour can be provided only via system app(which >> is now developing) for COPE situation. What kind of solution are we going >> to provide for BYOD scenario? >> 2. If we provide ABL, we need to restrict the application execution and >> installation. Again this will be feasible with COPE scenario because of the >> system app. But for BYOD scenario, according to posts I have read there is >> no broadcasts for application launch event or application install start >> events. So one option would be to create a periodically running background >> service which search for application that are running in the foreground and >> blocking that app if found in restriction list. WDYT about this approach? >> Anyway even via this approach, It is not possible to detect which >> application is installing at the moment of checking. In that case how we >> blocking app installation. Any idea to resolve this is much appreciated. >> >> Thanks >> >> On Mon, Feb 8, 2016 at 5:36 AM, Dilan Udara Ariyaratne <dil...@wso2.com> >> wrote: >> >>> Hi Lakshman, >>> >>> With respect to EMM space, I think that this requirement should be >>> handled from device policy level. >>> >>> FYI, a device policy is a set of configurations that we set to
Re: [Architecture] [EMM] Enabling application white listing and application blacklisting support
Hi All, After bit of discussion with EMM team, it is decided to move creation of white list policy to the app manager it self. We will provide a configuration whether white list is enabled or not. If white list enabled in the configuration, a new policy will created in EMM when new application is going to publish. All other later application will be added to existing white list policy when they are going to publish. This new policy will list in policy list and can be edit later to add a new policy criteria. When considering policy compliance, EMM will request application list with roles and update the existing policy and will request installed application list from device and will compare both for compliance Or another option would be to update the existing policy when something happen in app manager that would break existing app to role mapping. Ex: 1. Remove a role from a user 2. Remove a role from an application and request installed application list from device and check both for compliance. Another thing is these application restrictions will be implement based on COPE at initial stage. BYOD scenario will be considered later because of the complications mentioned in previous reply. WDYT about this approach? Thanks On Tue, Feb 23, 2016 at 3:26 PM, Lakshman Udayakantha <lakshm...@wso2.com> wrote: > Hi DilanA/EMM Team, > > @DilanA :Thanks for the information. > > I have assumed policy creator know the package names of the applications > which need to be restricted in the device and implemented the mdm policy UI > for app restriction list and able to publish the restriction list to the > device successfully as the first step. > > Some terminology has been changed after this thread initialised. As of now > If AWL is enabled we will provide role based application access. Policy > creator will define application white list with set of roles along with the > application. Only those roles will be able to access the application. If > ABL is enabled, policy creator will define black list via the UI and those > application list will not be allowed to run on any device. > > @EMM Team: I got several questions regarding the restriction apps using > mobile agent app. > 1. If we provide AWL, only those applications will show in app manager > store. Other app stores, side loading and google play store needs to be > blocked. This kind of behaviour can be provided only via system app(which > is now developing) for COPE situation. What kind of solution are we going > to provide for BYOD scenario? > 2. If we provide ABL, we need to restrict the application execution and > installation. Again this will be feasible with COPE scenario because of the > system app. But for BYOD scenario, according to posts I have read there is > no broadcasts for application launch event or application install start > events. So one option would be to create a periodically running background > service which search for application that are running in the foreground and > blocking that app if found in restriction list. WDYT about this approach? > Anyway even via this approach, It is not possible to detect which > application is installing at the moment of checking. In that case how we > blocking app installation. Any idea to resolve this is much appreciated. > > Thanks > > On Mon, Feb 8, 2016 at 5:36 AM, Dilan Udara Ariyaratne <dil...@wso2.com> > wrote: > >> Hi Lakshman, >> >> With respect to EMM space, I think that this requirement should be >> handled from device policy level. >> >> FYI, a device policy is a set of configurations that we set to be >> published for a number of devices based on Roles and Users. >> If we think about this requirement too in the same way, it is a >> application level configuration that we publish for a set of devices based >> on Roles and Users. >> >> Therefore, It seems that you can integrate this use case with the >> existing device policy UI [1] as two more feature additions to the >> "Configure Profile" section. >> i.e. One feature for White Listed Apps and the other for Black Listed >> Apps. >> >> Thanks, >> Dilan. >> >> >> *Dilan U. Ariyaratne* >> Software Engineer >> WSO2 Inc. <http://wso2.com/> >> Mobile: +94725197942 >> lean . enterprise . middleware >> >> On Tue, Feb 2, 2016 at 5:47 PM, Lakshman Udayakantha <lakshm...@wso2.com> >> wrote: >> >>> [adding Dakshika] >>> >>> On Tue, Feb 2, 2016 at 5:45 PM, Lakshman Udayakantha <lakshm...@wso2.com >>> > wrote: >>> >>>> Hi All, >>>> >>>> @KasunD/PrabathA: Thanks for your suggestions. I will check for methods >>>> to block app
Re: [Architecture] [EMM] Enabling application white listing and application blacklisting support
Hi DilanA/EMM Team, @DilanA :Thanks for the information. I have assumed policy creator know the package names of the applications which need to be restricted in the device and implemented the mdm policy UI for app restriction list and able to publish the restriction list to the device successfully as the first step. Some terminology has been changed after this thread initialised. As of now If AWL is enabled we will provide role based application access. Policy creator will define application white list with set of roles along with the application. Only those roles will be able to access the application. If ABL is enabled, policy creator will define black list via the UI and those application list will not be allowed to run on any device. @EMM Team: I got several questions regarding the restriction apps using mobile agent app. 1. If we provide AWL, only those applications will show in app manager store. Other app stores, side loading and google play store needs to be blocked. This kind of behaviour can be provided only via system app(which is now developing) for COPE situation. What kind of solution are we going to provide for BYOD scenario? 2. If we provide ABL, we need to restrict the application execution and installation. Again this will be feasible with COPE scenario because of the system app. But for BYOD scenario, according to posts I have read there is no broadcasts for application launch event or application install start events. So one option would be to create a periodically running background service which search for application that are running in the foreground and blocking that app if found in restriction list. WDYT about this approach? Anyway even via this approach, It is not possible to detect which application is installing at the moment of checking. In that case how we blocking app installation. Any idea to resolve this is much appreciated. Thanks On Mon, Feb 8, 2016 at 5:36 AM, Dilan Udara Ariyaratne <dil...@wso2.com> wrote: > Hi Lakshman, > > With respect to EMM space, I think that this requirement should be handled > from device policy level. > > FYI, a device policy is a set of configurations that we set to be > published for a number of devices based on Roles and Users. > If we think about this requirement too in the same way, it is a > application level configuration that we publish for a set of devices based > on Roles and Users. > > Therefore, It seems that you can integrate this use case with the existing > device policy UI [1] as two more feature additions to the "Configure > Profile" section. > i.e. One feature for White Listed Apps and the other for Black Listed Apps. > > Thanks, > Dilan. > > > *Dilan U. Ariyaratne* > Software Engineer > WSO2 Inc. <http://wso2.com/> > Mobile: +94725197942 > lean . enterprise . middleware > > On Tue, Feb 2, 2016 at 5:47 PM, Lakshman Udayakantha <lakshm...@wso2.com> > wrote: > >> [adding Dakshika] >> >> On Tue, Feb 2, 2016 at 5:45 PM, Lakshman Udayakantha <lakshm...@wso2.com> >> wrote: >> >>> Hi All, >>> >>> @KasunD/PrabathA: Thanks for your suggestions. I will check for methods >>> to block application installations for lower api level than 23 also. >>> I have created mockup UIs to create, edit , view lists which should be >>> added to app publisher UI and attached mockup UIs to this mail. >>> @UX team: Could you do a quick review and make suggestions to make them >>> better. >>> >>> >>> Thanks >>> >>> On Tue, Feb 2, 2016 at 9:54 AM, Harshan Liyanage <hars...@wso2.com> >>> wrote: >>> >>>> Hi Inosh, >>>> >>>> There may be some cases where enterprises need to have application >>>> policies for individual users. But I think that scenario is very unlikely. >>>> If we take an organization, every user will map to one or more user-roles. >>>> There might be situations where a role has only one user (i.e like CEO, >>>> MD). But still we can achieve it via the application policies for >>>> user-roles. >>>> >>>> Thanks, >>>> >>>> Harshan Liyanage >>>> Software Engineer >>>> Mobile: *+94724423048* >>>> Email: hars...@wso2.com >>>> Blog : http://harshanliyanage.blogspot.com/ >>>> *WSO2, Inc. :** wso2.com <http://wso2.com/>* >>>> lean.enterprise.middleware. >>>> >>>> On Tue, Feb 2, 2016 at 9:37 AM, Inosh Perera <ino...@wso2.com> wrote: >>>> >>>>> Hi all, >>>>> >>>>> Role based application restriction will be provided. Administrator >>>>
Re: [Architecture] [EMM] Enabling application white listing and application blacklisting support
[adding Dakshika] On Tue, Feb 2, 2016 at 5:45 PM, Lakshman Udayakantha <lakshm...@wso2.com> wrote: > Hi All, > > @KasunD/PrabathA: Thanks for your suggestions. I will check for methods to > block application installations for lower api level than 23 also. > I have created mockup UIs to create, edit , view lists which should be > added to app publisher UI and attached mockup UIs to this mail. > @UX team: Could you do a quick review and make suggestions to make them > better. > > > Thanks > > On Tue, Feb 2, 2016 at 9:54 AM, Harshan Liyanage <hars...@wso2.com> wrote: > >> Hi Inosh, >> >> There may be some cases where enterprises need to have application >> policies for individual users. But I think that scenario is very unlikely. >> If we take an organization, every user will map to one or more user-roles. >> There might be situations where a role has only one user (i.e like CEO, >> MD). But still we can achieve it via the application policies for >> user-roles. >> >> Thanks, >> >> Harshan Liyanage >> Software Engineer >> Mobile: *+94724423048* >> Email: hars...@wso2.com >> Blog : http://harshanliyanage.blogspot.com/ >> *WSO2, Inc. :** wso2.com <http://wso2.com/>* >> lean.enterprise.middleware. >> >> On Tue, Feb 2, 2016 at 9:37 AM, Inosh Perera <ino...@wso2.com> wrote: >> >>> Hi all, >>> >>> Role based application restriction will be provided. Administrator will >>> define a list of applications as a black list and a set of roles which is >>> to be restricted to the application, along with the applications. >>> Is there any particular reason for not having application policies for >>> individual users? >>> >>> Regards, >>> Inosh >>> >>> On Mon, Feb 1, 2016 at 11:05 PM, Prabath Abeysekera <praba...@wso2.com> >>> wrote: >>> >>>> >>>> On Mon, Feb 1, 2016 at 6:14 PM, Kasun Dananjaya Delgolla < >>>> kas...@wso2.com> wrote: >>>> >>>>> Hi Lakshman, >>>>> >>>>> In terms of Android you can use blocking APIs[1] in Marshmallow SDK >>>>> (SDK 23) to achieve this. We already use DevicePolicyManager API so you >>>>> can >>>>> straightaway add these new stuff into the same android agent API layer. >>>>> Also for older API levels ( < 23) earlier we used a mechanism just to warn >>>>> the user if a blacklisted app is installed on the device since blocking of >>>>> apps is not supported in those API levels. >>>>> >>>> >>>> We might need to dig slightly deep into some of the APIs around and see >>>> if we've already got anything to mimic what's done in DevicePolicyManager, >>>> which is part of Marshmallow SDK; in previous versions of Android SDK. So, >>>> please check if there's any mechanism that'd potentially allow us to go >>>> beyond merely warning the user when a blacklisted application is installed >>>> and then block the installation completely particularly targeting SDKs < >>>> 23. >>>> >>>> Cheers, >>>> Prabath >>>> >>>> >>>>> >>>>> One more thing, we can add this to the system app which I'm in the >>>>> process of building. Then we can enable COPE (rooted/system access >>>>> granted) >>>>> devices to blacklist/whitelist apps even though the API level is < 23. >>>>> >>>>> [1] - >>>>> http://developer.android.com/reference/android/app/admin/DevicePolicyManager.html >>>>> >>>>> Thanks >>>>> >>>>> On Mon, Feb 1, 2016 at 5:50 PM, Lakshman Udayakantha < >>>>> lakshm...@wso2.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> There is a requirement to implement application white listing and >>>>>> application black listing support in Enterprise Mobility Manager. >>>>>> Application white listing means creating a list of applications which are >>>>>> only allowed to run on mobile devices which are connected to EMM. >>>>>> Application blacklisting is the opposite meaning in which there is a list >>>>>> of applications which are only not allowed to run on mobile devices which >>>>>> connected to EMM. >>>>>> As a solution for this we thought to introduce
[Architecture] [EMM] Enabling application white listing and application blacklisting support
Hi, There is a requirement to implement application white listing and application black listing support in Enterprise Mobility Manager. Application white listing means creating a list of applications which are only allowed to run on mobile devices which are connected to EMM. Application blacklisting is the opposite meaning in which there is a list of applications which are only not allowed to run on mobile devices which connected to EMM. As a solution for this we thought to introduce a configuration to identify black listing, white listing enabled or not and exactly which listing is enabled and If each configuration enabled separately EMM will behave in following manner. If ABL enabled, Role based application restriction will be provided. Administrator will define a list of applications as a black list and a set of roles which is to be restricted to the application, along with the applications. If AWL enabled, Administrator will check specific list of applications from admin UI. Only these applications will load on app store. Other means of applications installing will be blocked. 1. Blocking side-loading. 2. Third party app store blocking except EMM app store. 3. Google Play app blocking Any suggestions and thoughts are highly appreciated. Thanks -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0714388124* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] [APIM] Using the registry lifecycle in the API Manager
Hi, API Manager currently uses its own lifecycle implementation. It will limited to a defined set of lifecycle statuses. With API Manager 1.10.0, The plan is to move with registry life cycles. Currently API Publisher loads pre-defined list of statuses. This will be changed to load statues from registry. Registry manages lifecycle information in APILifeCycle.xml. After loading lifecycle status from registry, it will be able to add custom lifecycle status also to API Manager. Here are two suggestions raised to implement this in UI level, 1. We can add a label to show the status instead of drop down list and can show the event transitions as buttons. updated status will show on status label. Those visible status names will load from status ids in APILifeCycle.xml. According to the status, appropriate buttons will be loaded. Visible button status will be transition event in APILifeCycle.xml. As an example: when API is in Created status the only possible status transitions are "Promote" and "Publish" according to default APILifeCycle.xml. So buttons with those names will be visible as below picture depicted. Pros: Invalid transitions will not be selected from drop down list, Easier to implement rather replace with API Manager 2.0.0 publisher feature lifecycle UI (look option 2). Cons :User has no way to know the next status until user clicks the promote or demote buttons. As a solution for this we can load the button names for next status. But if there are huge number of next statuses it will be unfeasible to load all the button names. 2. This has already implemented in API Manager 2.0.0 feature for greg integration with D3js library. We can integrate that UI in API Manager 1.10.0 too by replacing current drop down menu and button set. Pros : This will give a clear picture to the user about status transition and nice looking interface, Invalid transitions will not be selected from drop down list. Cons : Since future UIs of API Manager will be the UI belongs to API Manager 2.0.0 feature set, recreating API Manager 2.0.0 lifecycle UI in API Manager 1.10.0 will be an additional effort. Thanks -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Implementation of the getting the grouping id for application and subscription sharing
Hi All, The problem here is get the user grouping id when user logged in. Extracting grouping id is happened via a custom implementation by implementing the interface as mentioned in [1]. We have to implement the custom class to extract the grouping id and give that custom class reference in api-manager.xml. As per the offline discussion with Sanjeewa, I got several ways to implement this. 1. Using AbstractAxis2ConfigurationContextObserver class. When user logged into the system, it creates a carbon context. We can extend this class and override it’s methods to call our custom implementation at login time. 2. Using jaggery_acs.jag. When user logged into the system, it calls the jaggery_acs.jag file. So we can call the custom implementation during the execution of jaggery_acs.jag. Since the session variable is in jaggery layer, we decided to go with second option because after extracting the grouping id we thought to add it to session for later use like extracting subscriptions and applications. Appreciate your feedback on this. [1] Login Interceptor Interface of API Manager mail thread Thanks -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Login Interceptor Interface of API Manager
Hi All,
How about refactoring the interface like below.
public interface LoginPostExecutor {
public String getGroupingIdentifier(String response);
}
@Amila: I thought to call the implementation of that interface in
the jsFunction_login() method of StoreHostObject and assign the returned
grouping id to the current session. anyway I will check the feasibility of
using WorkflowExecutor
https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/apimgt/org.wso2.carbon.apimgt.impl/1.2.3/src/main/java/org/wso2/carbon/apimgt/impl/workflow/WorkflowExecutor.java
for
this task.
Thanks
On Tue, Jan 13, 2015 at 2:34 PM, Amila De Silva ami...@wso2.com wrote:
Hi Lakshman,
Can you explain a bit about when this is invoked, and for what tasks this
is going to be used? If we are trying to provide an extension point to run
a custom code after a user logs in, I think then we should try to use
WorkflowExecutor
https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/apimgt/org.wso2.carbon.apimgt.impl/1.2.3/src/main/java/org/wso2/carbon/apimgt/impl/workflow/WorkflowExecutor.java
first.
On Tue, Jan 13, 2015 at 1:22 PM, Lakshman Udayakantha lakshm...@wso2.com
wrote:
Hi All,
I am working on [1]. In that I have to implement the application and
subscription sharing capability based on the organisation of the login
user. I have to create a Login interceptor interface to determine the
organisation of login user. We have to extract the organisation identifier
from login response using login interceptor. below is a proposed login
interceptor interface.
public interface LoginInterceptor {
public String getOrganization(String response);
}
When using WSO2 identity server we can add organisation as an attribute
to SAML response and we can extract it in the implementation of the
interceptor in API Manager side. Appreciate your thought on this.
[1]https://redmine.wso2.com/issues/2917
--
Lakshman Udayakantha
WSO2 Inc. www.wso2.com
lean.enterprise.middleware
Mobile: *0711241005*
--
*Amila De Silva*
WSO2 Inc.
mobile :(+94) 775119302
--
Lakshman Udayakantha
WSO2 Inc. www.wso2.com
lean.enterprise.middleware
Mobile: *0711241005*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Login Interceptor Interface of API Manager
Hi All,
I am working on [1]. In that I have to implement the application and
subscription sharing capability based on the organisation of the login
user. I have to create a Login interceptor interface to determine the
organisation of login user. We have to extract the organisation identifier
from login response using login interceptor. below is a proposed login
interceptor interface.
public interface LoginInterceptor {
public String getOrganization(String response);
}
When using WSO2 identity server we can add organisation as an attribute to
SAML response and we can extract it in the implementation of the
interceptor in API Manager side. Appreciate your thought on this.
[1]https://redmine.wso2.com/issues/2917
--
Lakshman Udayakantha
WSO2 Inc. www.wso2.com
lean.enterprise.middleware
Mobile: *0711241005*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] API import/export
Hi all, We are developing an API import/export feature for API manager which has been discussed earlier as well in [1]. We have identified the following artifacts to be included in the exported file of an API for now. The archive file structure of exported APIs will be similar to below. -- pizzaShack - v1 - docs - image - sequences - meta-info The UI will be presented through the admin-dashboard of API manager where the available list of APIs will be displayed. The user will have the ability to select one or many APIs and create an archive with the selected APIs. After the archive is created, the user will be provided with a download link to download the archive. Please refer below image as the UI. [1] [Architecture] Export/import APIs? http://mail.wso2.org/mailarchive/architecture/2013-March/011049.html -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
