[Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
On Tue, Mar 24, 2015 at 8:14 PM, danush...@wso2.com wrote: I think same behavior is there in APIM too. They have the API list and separate list called Recently added. To get recently added to API list takes few seconds. Can’t we implement similar thing here? I don't see an issue to fix in indexing . There is no way to index the resource just after adding. Indexing will trigger from the frequency which scheduled in registry.xml file. So if you need reduce the indexing task frequency. -Ajith Thanks Regards Danushka Fernando Software Engineer WSO2 inc. http://wso2.com/ Mobile : +94716332729 *From:* Dimuthu Leelarathne dimut...@wso2.com *Sent:* Tuesday, March 24, 2015 10:22 PM *To:* Punnadi Gunarathna punn...@wso2.com *Cc:* architecture architecture@wso2.org Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Ajith Vitharana. WSO2 Inc. - http://wso2.org Email : aji...@wso2.com Mobile : +1-812-360-5335 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
I think same behavior is there in APIM too. They have the API list and separate list called Recently added. To get recently added to API list takes few seconds. Can’t we implement similar thing here? Thanks Regards Danushka Fernando Software Engineer WSO2 inc. http://wso2.com/ Mobile : +94716332729 From: Dimuthu Leelarathne Sent: Tuesday, March 24, 2015 10:22 PM To: Punnadi Gunarathna Cc: architecture Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3]https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi Ajith What I was pointing out was this could be a fix in our side (AF). Sorry if I mislead you. Thanks Regards Danushka Fernando Software Engineer WSO2 inc. http://wso2.com/ Mobile : +94716332729 On Mar 25, 2015 6:11 AM, Ajith Vitharana aji...@wso2.com wrote: On Tue, Mar 24, 2015 at 8:14 PM, danush...@wso2.com wrote: I think same behavior is there in APIM too. They have the API list and separate list called Recently added. To get recently added to API list takes few seconds. Can’t we implement similar thing here? I don't see an issue to fix in indexing . There is no way to index the resource just after adding. Indexing will trigger from the frequency which scheduled in registry.xml file. So if you need reduce the indexing task frequency. -Ajith Thanks Regards Danushka Fernando Software Engineer WSO2 inc. http://wso2.com/ Mobile : +94716332729 *From:* Dimuthu Leelarathne dimut...@wso2.com *Sent:* Tuesday, March 24, 2015 10:22 PM *To:* Punnadi Gunarathna punn...@wso2.com *Cc:* architecture architecture@wso2.org Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Ajith Vitharana. WSO2 Inc. - http://wso2.org Email : aji...@wso2.com Mobile : +1-812-360-5335 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi Dimuthu, So shall I go-ahead with indexing? On Tue, Mar 24, 2015 at 10:22 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi all, If we are to make use of indexing, we need to check the feasibility to index existing data as well because size of data in cloud setup is huge. However this is a one time task that we can find a solution such as copy the indexed data into server after running indexes offline for dump of live data. It seems that we need to come up with a aggregated solution of indexed data and non-indexed data(new additions) then removing duplicates etc. We can not depend on retries because we can not assume the time taken to index etc. thank you. On Wed, Mar 25, 2015 at 9:57 AM, Punnadi Gunarathna punn...@wso2.com wrote: Hi Dimuthu, So shall I go-ahead with indexing? On Tue, Mar 24, 2015 at 10:22 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi, We should not make architectural decision fearing a 29 hour data migration :). @Chandana and Greg team - If you guys say this is a fixed thing and cannot be fixed, we'll give priority to our Runtime version database and move ahead. :) We'll still keep meta data in registry. Punnadi and AF team - Lets give priority to our Runtime database because of the 10 second delay ( probably in cloud it will be 30 seconds) thanks, dimuthu On Wed, Mar 25, 2015 at 10:40 AM, Manjula Rathnayake manju...@wso2.com wrote: Hi all, If we are to make use of indexing, we need to check the feasibility to index existing data as well because size of data in cloud setup is huge. However this is a one time task that we can find a solution such as copy the indexed data into server after running indexes offline for dump of live data. It seems that we need to come up with a aggregated solution of indexed data and non-indexed data(new additions) then removing duplicates etc. We can not depend on retries because we can not assume the time taken to index etc. thank you. On Wed, Mar 25, 2015 at 9:57 AM, Punnadi Gunarathna punn...@wso2.com wrote: Hi Dimuthu, So shall I go-ahead with indexing? On Tue, Mar 24, 2015 at 10:22 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi, I wasn't suggesting to index the resource as soon as it is created, but rather whether it is possible to fix the 10 second delay that we have to introduce between creating and reading. artifactManager.createResource() //wait 10 seconds --- This is a problem artifactManager.readResource() @Punnadi - Please stop with indexing for now. Because if we are fixing it we need to fix without retries and needs design discussion. Indexing would only fix the initial loading time. Put it on hold until we do a detail design review. thanks, dimuthu On Wed, Mar 25, 2015 at 9:57 AM, Punnadi Gunarathna punn...@wso2.com wrote: Hi Dimuthu, So shall I go-ahead with indexing? On Tue, Mar 24, 2015 at 10:22 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi Dimuthu, As explained by Ajith as well, this is the expected behavior. However, you can optimize your indexing related parameters (indexingFrequencyInSeconds, etc.) according to your requirements and available resources. Regards, Chandana On Wed, Mar 25, 2015 at 10:51 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi, We should not make architectural decision fearing a 29 hour data migration :). @Chandana and Greg team - If you guys say this is a fixed thing and cannot be fixed, we'll give priority to our Runtime version database and move ahead. :) We'll still keep meta data in registry. Punnadi and AF team - Lets give priority to our Runtime database because of the 10 second delay ( probably in cloud it will be 30 seconds) thanks, dimuthu On Wed, Mar 25, 2015 at 10:40 AM, Manjula Rathnayake manju...@wso2.com wrote: Hi all, If we are to make use of indexing, we need to check the feasibility to index existing data as well because size of data in cloud setup is huge. However this is a one time task that we can find a solution such as copy the indexed data into server after running indexes offline for dump of live data. It seems that we need to come up with a aggregated solution of indexed data and non-indexed data(new additions) then removing duplicates etc. We can not depend on retries because we can not assume the time taken to index etc. thank you. On Wed, Mar 25, 2015 at 9:57 AM, Punnadi Gunarathna punn...@wso2.com wrote: Hi Dimuthu, So shall I go-ahead with indexing? On Tue, Mar 24, 2015 at 10:22 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- *Chandana Napagoda* Senior Software Engineer WSO2 Inc. - http://wso2.org *Email : chand...@wso2.com chand...@wso2.com**Mobile : +94718169299* *Blog :http://cnapagoda.blogspot.com http://cnapagoda.blogspot.com* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi, On Wed, Mar 25, 2015 at 5:21 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi, We should not make architectural decision fearing a 29 hour data migration :). @Chandana and Greg team - If you guys say this is a fixed thing and cannot be fixed, we'll give priority to our Runtime version database and move ahead. :) We'll still keep meta data in registry. Punnadi and AF team - Lets give priority to our Runtime database because of the 10 second delay ( probably in cloud it will be 30 seconds) Please note - Solr is for content indexing. We will use DB indexing. We don't need to do content indexing. thanks, dimuthu thanks, dimuthu On Wed, Mar 25, 2015 at 10:40 AM, Manjula Rathnayake manju...@wso2.com wrote: Hi all, If we are to make use of indexing, we need to check the feasibility to index existing data as well because size of data in cloud setup is huge. However this is a one time task that we can find a solution such as copy the indexed data into server after running indexes offline for dump of live data. It seems that we need to come up with a aggregated solution of indexed data and non-indexed data(new additions) then removing duplicates etc. We can not depend on retries because we can not assume the time taken to index etc. thank you. On Wed, Mar 25, 2015 at 9:57 AM, Punnadi Gunarathna punn...@wso2.com wrote: Hi Dimuthu, So shall I go-ahead with indexing? On Tue, Mar 24, 2015 at 10:22 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Fac] Performance Improvements - Issues in enabling GReg resource Indexing in App Factory
Hi Chandana, Could we get fixes from Registry side for this? thanks, dimuthu On Tue, Mar 24, 2015 at 7:29 PM, Punnadi Gunarathna punn...@wso2.com wrote: Hi All, While fixing a cloud bug which is AF code tries to retrieve information of irrelevant apps when inside an app [1] , I found that passing a Map object rather than GenericArtifactFilter object to GenericArtifactManager would solve the problem [2]. When I changed the GenericArtifactFilter object to a Map object [3], it threw some exceptions. With the help of Greg team it was found out that the App Factory resources weren't indexed which result those exceptions. So I indexed the resource [5]. But after applying indexing I'm facing more problems as follows: 1. When an application is created, the 'trunk' created via 'appversion.rxt' will not be returned from getAppVersionRXTFromRegistry method in RxtManager.java until for few seconds (such as 10s), So it requires to implement retry attempts until it returns the version 'trunk' to proceed further in app creation process. 2. When an app version is created, newly created verson will not be returned by the same method mentioned in 1, due to the same reason and it will require some code changes to fix the exceptions introduced later in execution path. Moreover this might be a significant issue in Cloud as well. Please advice whether to keep the old implementation or to godhead with indexing and sort the remaining issues. [1] https://wso2.org/jira/browse/APPFAC-3009 [2][Dev] [Greg] What is the difference between overloaded findGenericArtifacts method in GenericArtifactManager - mail [3] https://github.com/punnadih/product-af/commit/dff7920619d25a134daef8f0134744908ae1e402 [4][Dev][App Fac] Registry resources Indexing issue fixed - mail -- Thanks and Regards, Punnadi Gunarathna Senior Software Engineer, WSO2, Inc.; http://wso2.com http://wso2 Blog: http://hi-my-world.blogspot.com/ Tel : 94 11 214 5345 Fax :94 11 2145300 http://lalajisureshika.blogspot.com/ -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
