Hi all, I'm testing the registry with 10,1000,10,000 Experiments and I've tested the database performance executing the getAllExperiments method. I'll post the complete analysis.
What are the other methods that I should test using? getExperiment(experiment_id) searchExperiment Any pointers? On Wed, Jul 23, 2014 at 6:07 PM, Marlon Pierce <[email protected]> wrote: > Thanks, Sachith. Did you look at scaling also? That is, will the > operations below still be the slowest if the DB is 10x, 100x, 1000x bigger? > > Marlon > > > On 7/23/14, 8:22 AM, Sachith Withana wrote: > >> Hi all, >> >> I'm profiling the current registry in few different aspects. >> >> I looked into the database operations and I've listed the operations that >> take the most amount of time. >> >> 1. Getting the Status of an Experiment (takes around 10% of the overall >> time spent) >> Has to go through the hierarchy of the datamodel to get to the actual >> experiment status ( node, tasks ...etc) >> >> 2. Dealing with the Application Inputs >> Strangely it takes a long time for the queries regarding the >> ApplicationInputs to complete. >> This is a part of the new Application Catalog >> >> 3. Getting all the Experiments ( using the * wild card) >> This takes the maximum amount of time when queried at first. But >> thanks >> to the OpenJPA caching, it flattens out as we keep querying. >> >> To reduce the first issue, I would suggest to have a different table for >> Experiment Summaries, >> where the status ( both the state and the state update time) would be the >> only varying entity, and use that to improve the query time for Experiment >> summaries. >> >> It would also help improve the performance for getting all the Experiments >> ( experiment summaries) >> >> WDYT? >> >> ToDos : Look into memory consumption ( in terms of memory leakage ...etc) >> >> >> Any more suggestions? >> > > -- Thanks, Sachith Withana
