+1 .. great summary. Learn to take notes properly guys .. giving the big picture and context is critical for people not in the room to be able to grok the conversation.
On Fri, Aug 9, 2013 at 9:33 AM, Srinath Perera <[email protected]> wrote: > Shelan, we need to tell the big picture and provide details so others can > get what we discussed. I know it is not easy, but we have to try! > > Let me try > > Greg use Solr to index and support property based search and sorted > results, and it is very fast. Now overall experience is very smooth. > > There were three main concerns > 1) We periodically poll the registry, get the activity log (use timestamp > to decide where to begin the next batch) and index them. However, if there > are many activities under the same timestamp, there is a chance we might > miss some of them. This is because we store the last processed timestamp > and get next batch from the next timestamp. This is the issue Senaka > provides a solution in mail before. > > 2) Solr is heavy, so we have to make sure it does not interfere with the > registry normal operations when it run index. So we need to test the > performance of registry over time (1hours) and make sure it does not slow > down when solr ran. > > 3) When user added resources, they only turn up in the search after being > indexed by periodic loop. We decided this is OK. > > --Srinath > > > > > On Thu, Aug 8, 2013 at 9:21 PM, Senaka Fernando <[email protected]> wrote: > >> Hi all, >> >> We happened to take a look @ the REG_LOG table. There is a separate ID >> which is used for the primary key. However, this ID is not exposed from the >> APIs. Therefore, if we are to use something other than the timestamp, then >> we'll have to do an API change. So, we decided to report a JIRA (Ajith will >> do that), and fix it for an upcoming release. In the meantime, we'll have >> to document the limitation. Shelan I think that's the only thing you missed >> out. I believe that we can safely mark the redmine issue as closed provided >> that the integration tests are covering good ground. Ajith please have a >> look into that. >> >> Thanks, >> Senaka. >> >> >> On Thu, Aug 8, 2013 at 5:31 PM, Shelan Perera <[email protected]> wrote: >> >>> Solr integration / scalability >>> >>> Clock resolution does not happen for same time activity logging. >>> >>> Possibility of using an counter to solve the problem with missing >>> activities for same time stamp. But not advisable due to data migration >>> with the schema changes. >>> >>> Possibility of running Solr as an external server >>> >>> Please add if there is anything missing. >>> >>> When you run the performance test , Get not only the average but look >>> for serious deviations/spikes etc. >>> >>> Thanks >>> >>> -- >>> *Shelan Perera* >>> >>> Senior Software Engineer >>> ** >>> Integration Technology Group >>> *WSO2, Inc. : wso2.com* >>> lean.enterprise.middleware. >>> >>> *Blog* : blog.shelan.org >>> *Linked-i*n : http://www.linkedin.com/pub/shelan-perera/a/194/465 >>> *Twitter* : https://twitter.com/#!/shelan >>> >>> *Mobile* : +94 772 604 402 >>> >>> >> >> >> -- >> * <http://us13.wso2con.com/> >> * >> * >> * >> *Senaka Fernando* >> Senior Technical Lead; WSO2 Inc.; http://wso2.com* >> Member; Apache Software Foundation; http://apache.org >> >> E-mail: senaka AT wso2.com >> **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818 >> Linked-In: http://linkedin.com/in/senakafernando >> >> *Lean . Enterprise . Middleware >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > ============================ > Srinath Perera, Ph.D. > http://people.apache.org/~hemapani/ > http://srinathsview.blogspot.com/ > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Sanjiva Weerawarana, Ph.D. Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ email: [email protected]; phone: +94 11 763 9614; cell: +94 77 787 6880 | +1 650 265 8311 blog: http://sanjiva.weerawarana.org/ Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
