For me the system tests are not much faster compared to trunk. Now at r23912, the disp_spin_speed branch takes ~216 seconds for all Relax_disp system tests. But for the trunk it is ~233 seconds. From your profiling script it's pretty clear though that there are insane speed ups happening, but it's not reflected so much in the system tests. It might be interesting to see what is causing these system tests to take so long - it's clear that it's not the target functions.
Regards, Edward On 13 June 2014 10:46, Troels Emtekær Linnet <[email protected]> wrote: > Yeahaa! > > I realised, that if I wanted to have the systemtests go faster, > I should kill this one. > > It is nice to see, that systemtest suite just runs faster and faster, the > more models > i kill. > > > > 2014-06-13 10:37 GMT+02:00 Edward d'Auvergne <[email protected]>: >> >> Now this code might be too fast for the user - they may not be fast >> enough to even see it in the log files ;) >> >> Regards, >> >> Edward >> >> >> On 13 June 2014 10:10, <[email protected]> wrote: >> > Author: tlinnet >> > Date: Fri Jun 13 10:10:04 2014 >> > New Revision: 23908 >> > >> > URL: http://svn.gna.org/viewcvs/relax?rev=23908&view=rev >> > Log: >> > Made the NOREX model af faster numpy array calculation. >> > >> > Task #7807 (https://gna.org/task/index.php?7807): Speed-up of dispersion >> > models for Clustered analysis. >> > >> > Modified: >> > branches/disp_spin_speed/target_functions/relax_disp.py >> > >> > Modified: branches/disp_spin_speed/target_functions/relax_disp.py >> > URL: >> > http://svn.gna.org/viewcvs/relax/branches/disp_spin_speed/target_functions/relax_disp.py?rev=23908&r1=23907&r2=23908&view=diff >> > >> > ============================================================================== >> > --- branches/disp_spin_speed/target_functions/relax_disp.py >> > (original) >> > +++ branches/disp_spin_speed/target_functions/relax_disp.py Fri Jun >> > 13 10:10:04 2014 >> > @@ -394,8 +394,11 @@ >> > if model == MODEL_NS_MMQ_3SITE_LINEAR: >> > self.func = self.func_ns_mmq_3site_linear >> > >> > + >> > # Setup special numpy array structures, for higher dimensional >> > computation. >> > - if model in [MODEL_B14, MODEL_B14_FULL, MODEL_CR72, >> > MODEL_CR72_FULL, MODEL_DPL94, MODEL_TAP03, MODEL_TP02, MODEL_TSMFK01]: >> > + test_models = [MODEL_B14, MODEL_B14_FULL, MODEL_CR72, >> > MODEL_CR72_FULL, MODEL_DPL94, MODEL_TAP03, MODEL_TP02, MODEL_TSMFK01] >> > + >> > + if model in test_models + [MODEL_NOREX]: >> > # Get the shape of back_calc structure. >> > # If using just one field, or having the same number of >> > dispersion points, the shape would extend to that number. >> > # Shape has to be: [ei][si][mi][oi]. >> > @@ -443,8 +446,11 @@ >> > self.r20b_struct = deepcopy(zeros_a) >> > self.dw_struct = deepcopy(zeros_a) >> > >> > - # Extract the frequencies to numpy array. >> > - self.frqs_a = multiply.outer( >> > asarray(self.frqs).reshape(self.NE, self.NS, self.NM), self.no_nd_struct ) >> > + # Setup special numpy array structures, for higher dimensional >> > computation. >> > + if model in test_models + [MODEL_NOREX]: >> > + if model in test_models: >> > + # Extract the frequencies to numpy array. >> > + self.frqs_a = multiply.outer( >> > asarray(self.frqs).reshape(self.NE, self.NS, self.NM), self.no_nd_struct ) >> > >> > if model in MODEL_LIST_CPMG_FULL: >> > self.cpmg_frqs_a = deepcopy(ones_a) >> > @@ -476,7 +482,7 @@ >> > # Extract number of dispersion points. >> > num_disp_points = >> > self.num_disp_points[ei][si][mi][oi] >> > >> > - if model in MODEL_LIST_CPMG_FULL: >> > + if model in MODEL_LIST_CPMG_FULL and model >> > != MODEL_NOREX: >> > # Extract cpmg_frqs and num_disp_points >> > from lists. >> > >> > self.cpmg_frqs_a[ei][si][mi][oi][:num_disp_points] = >> > self.cpmg_frqs[ei][mi][oi] >> > >> > self.num_disp_points_a[ei][si][mi][oi][:num_disp_points] = >> > self.num_disp_points[ei][si][mi][oi] >> > @@ -497,7 +503,7 @@ >> > self.power_a[ei][si][mi][oi][di] = >> > int(round(self.cpmg_frqs[ei][mi][0][di] * self.relax_times[ei][mi])) >> > self.tau_cpmg_a[ei][si][mi][oi][di] >> > = 0.25 / self.cpmg_frqs[ei][mi][0][di] >> > # For R1rho data. >> > - if model in MODEL_LIST_R1RHO_FULL: >> > + if model in MODEL_LIST_R1RHO_FULL and >> > model != MODEL_NOREX: >> > >> > self.disp_struct[ei][si][mi][oi][di] = 1.0 >> > >> > # Extract the frequencies to numpy >> > array. >> > @@ -511,11 +517,12 @@ >> > else: >> > >> > self.num_disp_points_a[ei][si][mi][oi][di] = 0 >> > >> > + if model in test_models: >> > + # Pre calculate frqs structure >> > + self.frqs_struct = self.frqs_a * self.disp_struct >> > + >> > # Make copy of values structure. >> > self.back_calc_a = deepcopy(self.values_a) >> > - >> > - # Pre calculate frqs structure >> > - self.frqs_struct = self.frqs_a * self.disp_struct >> > >> > # Find the numpy mask, which tells where values should be >> > replaced. >> > self.mask_replace_blank = masked_equal(missing_a, 1.0) >> > @@ -1465,34 +1472,16 @@ >> > # Unpack the parameter values. >> > R20 = params >> > >> > - # Initialise. >> > - chi2_sum = 0.0 >> > - >> > - # Loop over the experiment types. >> > - for ei in range(self.num_exp): >> > - # Loop over the spins. >> > - for si in range(self.num_spins): >> > - # Loop over the spectrometer frequencies. >> > - for mi in range(self.num_frq): >> > - # The R20 index. >> > - r20_index = mi + si*self.num_frq >> > - >> > - # Loop over the offsets. >> > - for oi in range(self.num_offsets[ei][si][mi]): >> > - # The R2eff values as R20 values. >> > - for di in >> > range(self.num_disp_points[ei][si][mi][oi]): >> > - self.back_calc[ei][si][mi][oi][di] = >> > R20[r20_index] >> > - >> > - # For all missing data points, set the >> > back-calculated value to the measured values so that it has no effect on >> > the >> > chi-squared value. >> > - for di in >> > range(self.num_disp_points[ei][si][mi][oi]): >> > - if self.missing[ei][si][mi][oi][di]: >> > - self.back_calc[ei][si][mi][oi][di] = >> > self.values[ei][si][mi][oi][di] >> > - >> > - # Calculate and return the chi-squared value. >> > - chi2_sum += chi2(self.values[ei][si][mi][oi], >> > self.back_calc[ei][si][mi][oi], self.errors[ei][si][mi][oi]) >> > + # Reshape R20 to per experiment, spin and frequency. >> > + self.back_calc_a[:] = multiply.outer( R20.reshape(self.NE, >> > self.NS, self.NM), self.no_nd_struct ) >> > + >> > + ## For all missing data points, set the back-calculated value >> > to the measured values so that it has no effect on the chi-squared value. >> > + if self.has_missing: >> > + # Replace with values. >> > + self.back_calc_a[self.mask_replace_blank.mask] = >> > self.values_a[self.mask_replace_blank.mask] >> > >> > # Return the total chi-squared value. >> > - return chi2_sum >> > + return chi2_rankN(self.values_a, self.back_calc_a, >> > self.errors_a) >> > >> > >> > def func_ns_cpmg_2site_3D(self, params): >> > >> > >> > _______________________________________________ >> > relax (http://www.nmr-relax.com) >> > >> > This is the relax-commits mailing list >> > [email protected] >> > >> > To unsubscribe from this list, get a password >> > reminder, or change your subscription options, >> > visit the list information page at >> > https://mail.gna.org/listinfo/relax-commits >> >> _______________________________________________ >> relax (http://www.nmr-relax.com) >> >> This is the relax-devel mailing list >> [email protected] >> >> >> To unsubscribe from this list, get a password >> reminder, or change your subscription options, >> visit the list information page at >> https://mail.gna.org/listinfo/relax-devel > > _______________________________________________ relax (http://www.nmr-relax.com) This is the relax-devel mailing list [email protected] To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel
