The attached file provides evaluation on Linux of all test cases run.
The pieces below the "----------------" separator all are expected
things due to tolerance differences, test system fragility, and so
forth. There are 7 scripts above that line, and they look acceptable to
me; see below.
/*Short summary:
*/
I think we are good to go on Linux, assuming the current test cases are
a fair evaluation of the R2016a system.
Caveat: Test cases were committed after this run was launched. I'm not
reporting on those cases in this evaluation.
/*Slightly longer evaluation:*///
These are the cases that might warrant a second set of eyes to evaluate:
*Script errors:**
*
* EphemPropagator_Code500_EphemCompare_LeapSecond'
o Output file name does not match .tc file
* EphemPropagator_Code500_ForwardProp_LeapSecond_Interp
o Output file name does not match .tc file
* TrkFile_SimEst_DSN_DS_Range_Solve_Bias_apriori
o Invalid scripting
* TrkFile_SimEst_GPS_ECFVec
o GPS test case not yet implemented
*Comparator errors:**
*
* EphemPropagator_Code500_ForwardProp_LeapSecond
o Prop outside of ephem span?
*Truth failures:**
*
* StringFuncs_Sprintf_Length_L
o Returned nan's (Issue - '%*L*f' on Linux)
* TrackingFileSet_RampTable_EOF
o Looks okay
- DJC (Cross eyed from reading test case reports and running scripts
that need running...)
On 09/16/2016 07:40 AM, Darrel J. Conway wrote:
Results from last night's run
--
Darrel J. Conway, Ph.D. Thinking Systems, Inc.
Senior Scientist and CEO 6441 N Camino Libby
Phone: (623) 298-4530 Tucson, AZ 85718
FAX: (520) 232-2533 www.thinksysinc.com
Cell: (520) 425-3626 d...@thinksysinc.com
Script errors:
EphemPropagator_Code500_EphemCompare_LeapSecond', ... <-- Output
file name does not match .tc file
EphemPropagator_Code500_ForwardProp_LeapSecond_Interp', ... <-- Output
file name does not match .tc file
TrkFile_SimEst_DSN_DS_Range_Solve_Bias_apriori', ... <--
Invalid scripting
TrkFile_SimEst_GPS_ECFVec', ... <-- GPS
test case not yet implemented
Comparator errors:
EphemPropagator_Code500_ForwardProp_LeapSecond', ... <-- Prop
outside of ephem span?
Truth failures:
StringFuncs_Sprintf_Length_L [#1 failed] <--
Returned nan's (Issue - '%Lf' on Linux)
TrackingFileSet_RampTable_EOF [#1 failed] <-- Looks
okay
-------------------------------------------------------------------------------
Truth Update Needed
Thruster_Validation_Tank_DifferentLengthThanMixtureRatio1 <-- Wrong
comparator; as expected, warning now thrown
Tolerance: Acceptable
CelestialBody_Earth_EopFileName [max diff 1.75759e-09 > tol 1e-09max
diff 1e-09 > tol ]
CelestialBody_Earth_EopFileName_Default [max diff 1.75144e-09 > tol
1e-09max diff 1e-09 > tol ]
CelestialBody_Earth_EopFileName_EmtpyString [max diff 1.75144e-09 > tol
1e-09max diff 1e-09 > tol ]
CoordinateSystem_VNBLocalvsBuiltIn [diff 5.28022e-06, tol 1e-06]
EphemPropagator_Code500_CentralBody_Jupiter [pos err 1.32294e-05, tol
3e-06] [GMT-5795]
EphemPropagator_Code500_CentralBody_Saturn [pos err 7.96186e-06, tol
3e-06] [GMT-5795]
EphemPropagator_Code500_ForwardProp_Interp [pos err 7.99517e-06, tol
3e-06] [GMT-5760]
GMAT_STM_Truncation_20x20 [diff 0.000238065, tol 1e-06]
GMAT_SunSync_EGM96_360x360 [pos err 3.48673e-05, tol 1e-05]
report_Spacecraft_Attached_hardware_Fuel_Tank_Properties_Impulsive_Burn
[diff 2e-05, tol 1e-05]
Simulator_Propagator_EarthPointMass_Drag/Simulator_Propagator_EarthPointMass_Drag_VelErr.report
[diff 2.06717e-05, tol 1e-05] [GMT-5417]
Simulator_Propagator_EarthPointMass_SRP_Drag/Simulator_Propagator_EarthPointMass_SRP_Drag_VelErr.report
[diff 3.00122e-05, tol 1e-05] [GMT-5417]
Tolerance: Check
Simulator_Propagator_EarthPointMass_Drag/Simulator_Propagator_EarthPointMass_Drag_PosErr.report
[diff 0.0207037, tol 1e-05] [GMT-5417]
Simulator_Propagator_EarthPointMass_SRP_Drag/Simulator_Propagator_EarthPointMass_SRP_Drag_PosErr.report
[diff 0.0299541, tol 1e-05] [GMT-5417]
GMATFunction_Function_Inside_While_If_Else_Loop_Global_SC_Subscribers_Other_Objects
[diff 0.0581503, tol 0.0005]
GMATFunction_LEOStationKeeping_Targeting_LocalSC_In_LocalSC_Out_Method_1 [diff
0.0604878, tol 1e-10]
GMATFunction_LEOStationKeeping_Targeting_LocalSC_In_LocalSC_Out_Method_2 [diff
0.0604878, tol 1e-10]
Tolerance: Why isn't the observed diff reported??? (looks ok)
GMAT_IncludeMacro_IncludeFile_Inside_For_Loop
<-- cm level agreement
GMAT_IncludeMacro_MarsBPlaneTarget_All_ResourcesCommands_From_IncludeScript',
...
GMAT_IncludeMacro_MarsBPlaneTarget_OnlyCommands_From_IncludeFile', ...
GMAT_IncludeMacro_MarsBPlaneTarget_OnlyResourceObjects_From_IncludeFile', ...
GMAT_IncludeMacro_Read_ThrusterAndFuelTank_From_IncludeFile', ...
GMAT_IncludeMacro_ReadSpacecraft_Update_to_NewState', ...
Tolerance: Known Test System Issue
Converge to different solution
Events_TESS_Optimize_TwoStage_FromLaunch_master_03b4b32/Events_ContactLocator_TESS.report
[start diff = 4793.312001s (event 24)]
Events_TESS_Optimize_TwoStage_FromLaunch_master_03b4b32/Events_EclipseLocator_TESS.report
[start diff = 4271.149002s (earth/10)]
CCSDS Ephem Comparator
Ephem_GMAT_CCSDS_OEM_60sec', ...
Ephem_GMAT_CCSDS_OEM_60sec_backward', ...
Ephem_GMAT_CCSDS_OEM_60sec_for_loop', ...
Ephem_GMAT_CCSDS_OEM_60sec_if_loop', ...
Ephem_GMAT_CCSDS_OEM_60sec_ifelse1_loop', ...
Ephem_GMAT_CCSDS_OEM_60sec_ifelse2_loop', ...
Ephem_GMAT_CCSDS_OEM_60sec_IterativeProcess_Optimizer', ...
Ephem_GMAT_CCSDS_OEM_60sec_target', ...
Ephem_GMAT_CCSDS_OEM_60sec_while_loop', ...
Ephem_GMAT_CCSDS_OEM_60sec_with_1DV_IB', ...
Ephem_GMAT_CCSDS_OEM_60sec_with_600secFB', ...
Ephem_GMAT_CCSDS_OEM_3600sec', ...
Ephem_GMAT_CCSDS_OEM_86400sec_IterativeProcess_Target_and_DynamicsChange', ...
Ephem_GMAT_CCSDS_OEM_DefaultIntegrationStep_with_1DV_IB', ...
Ephem_GMAT_CCSDS_OEM_ForwardProp_BackrProp_Monotonic_time', ...
Ephem_GMAT_InitialEpoch_FinalEpoch_StepSize', ...
EphemStressTests_Default_and_ABM_Integrator_CCSDS_60Sec_MultipleSCs_ForwardDirection',
...
EphemStressTests_Default_Prop_CCSDS_60Sec_MixedStopTypes_LogicTypes_DiscontinuityTypes_ForwardDirection',
...
EphemStressTests_Default_Prop_CCSDS_60Sec_MixedStopTypes_LogicTypes_DiscontinuityTypes_SolverTypes_ForwardDirect',
...
EphemStressTests_Default_Prop_CCSDS_60Sec_MixedStopTypes_LogicTypes_ForwardDirection',
...
EphemStressTests_Default_Prop_CCSDS_Forward_and_BackwardDirect', ...
EphemStressTests_PD78_ABM_Prop_CCSDS_60Sec_Step_ForwardDirection', ...
EphemStressTests_SPK_Integrator_CCSDS_60Sec_ForwardDirection', ...
GMAT_ToggleEphemeris_MiddleOff', ...
GMAT_ToggleEphemeris_MiddleOffThenOn', ...
GMAT_ToggleEphemeris_StartOffToggleOn', ...
GMAT_ToggleEphemeris_TwoEphemeris', ...
GMATFunction_Global_EphemerisFile', ...
GMATFunction_Global_EphemerisFile_With_Toggle', ...
GMATFunction_Global_EphemerisFile_With_Toggle_2', ...
GMATFunction_Hohmann_Targeter_Inside
Function_PassLocal_SC_In_Receive_LocalSC_Out_EphemerisFile', ...
Exact match and 3 digit exponent vs 2 digit
Ephem_GMAT_STKEphem_DefaultMission_FixedSteps_SimpleManeuver', ...
Ephem_GMAT_STKEphem_DefaultMission_FixedSteps_SimpleProp', ...
Ephem_GMAT_STKEphem_DefaultMission_IntegratorSteps_SimpleManeuver', ...
Ephem_GMAT_STKEphem_DefaultMission_IntegratorSteps_SimpleProp', ...
Ephem_GMAT_STKEphem_LEOStationKeeping_FixedSteps', ...
Ephem_GMAT_STKEphem_LEOStationKeeping_IntegratorSteps', ...
Ephem_GMAT_STKEphem_Mars_B_Plane_Targeting_FixedSteps', ...
Ephem_GMAT_STKEphem_Mars_B_Plane_Targeting_IntegratorSteps', ...
Ephem_GMAT_STKEphem_SimpleOrbitTransfer_FixedSteps', ...
Ephem_GMAT_STKEphem_SimpleOrbitTransfer_IntegratorSteps', ...
Ephem_GMAT_STKEphem_Target_Finite_Burn_to_Raise_Apogee_FixedSteps', ...
Ephem_GMAT_STKEphem_Target_Finite_Burn_to_Raise_Apogee_IntegratorSteps', ...
Write_Array_LogFile_Script [#2 failed]
Write_Array_ReportFile_Script [#2 failed]
Write_ImpulsiveBurn_LogFile_Concise [#9 failed]
Write_ImpulsiveBurn_LogFile_Script [#10 failed]
Write_ImpulsiveBurn_LogFile_Verbose [#10 failed]
Write_ImpulsiveBurn_ReportFile_Concise [#9 failed]
Write_ImpulsiveBurn_ReportFile_Script [#10 failed]
Write_ImpulsiveBurn_ReportFile_Verbose [#9 failed]
Write_Mode_Function [#17 failed]
Write_Multiple_LogFile_Concise [#24 failed]
Write_Multiple_LogFile_Concise_2 [#24 failed]
Write_Multiple_LogFile_Script [#2 failed] [GMT-5793]
Write_Multiple_LogFile_Script_2 [#2 failed] [GMT-5793]
Write_Multiple_LogFile_Verbose [#26 failed]
Write_Multiple_LogFile_Verbose_2 [#26 failed]
Write_Multiple_ReportFile_Concise [#24 failed]
Write_Multiple_ReportFile_Concise_2 [#24 failed] [GMT-5801]
Write_Multiple_ReportFile_Script [#2 failed] [GMT-5793]
Write_Multiple_ReportFile_Script_2 [#2 failed] [GMT-5793]
Write_Multiple_ReportFile_Verbose [#24 failed]
Write_Multiple_ReportFile_Verbose_2 [#24 failed] [GMT-5801]
Write_Propagator_LogFile_Concise [#14 failed]
Write_Propagator_LogFile_Script [#16 failed]
Write_Propagator_LogFile_Verbose [#15 failed]
Write_Propagator_ReportFile_Concise [#14 failed]
Write_Propagator_ReportFile_Script [#16 failed]
Write_Propagator_ReportFile_Verbose [#14 failed]
Write_Spacecraft_LogFile_Concise [#17 failed]
Write_Spacecraft_LogFile_Script [#17 failed] [GMT-5802]
Write_Spacecraft_LogFile_Verbose [#18 failed]
Write_Spacecraft_ReportFile_Concise [#17 failed]
Write_Spacecraft_ReportFile_LogFile_Script [#17 failed]
Write_Spacecraft_ReportFile_LogFile_Script [#17 failed]
Write_Spacecraft_ReportFile_Script [#17 failed]
Write_Spacecraft_ReportFile_Script_Include [#17 failed]
Write_Spacecraft_ReportFile_Script_RunEstimator [#17 failed]
Write_Spacecraft_ReportFile_Verbose [#17 failed]
StringFuncs_Sprintf_Flags_PlusSign [#1 failed]
StringFuncs_Sprintf_InlineFormatSpec [#1 failed]
Platform differences
SNOPT file format
SNOPT_OverrideSpecsFile [#1 failed]
SNOPT_SpecsFile [#1 failed]
Explicit line ending issues
BatchEstimator_Validation_Mode', ...
Bad tests: Prop small differences and expecting exact matches
CommandMode_ReportFile_Tests Delimiter and FixedWidth_Comma option', ...
CommandMode_ReportFile_Tests Delimiter and FixedWidth_Semicolon
option', ...
CommandMode_ReportFile_Tests Delimiter and FixedWidth_Space option', ...
CommandMode_ReportFile_Tests Delimiter and FixedWidth_Tab option', ...
EphemPropagator_Code500_EphemCompare [#1458 failed]
EphemPropagator_Code500_EphemCompare_Moon [#457 failed]
EphemPropagator_Code500_EphemCompare_Sun [#89 failed]
ReportFile_Delimiter_Comma', ...
ReportFile_Delimiter_SemiColon', ...
ReportFile_Delimiter_Space', ...
ReportFile_Delimiter_Tab', ...
Some things are just okay on *nix...
ScriptLanguage_Validation_CommandLabels_Report_026 [no error or
exception]
ScriptLanguage_Validation_CommandLabels_Toggle_026 [no error or
exception]
ScriptLanguage_Validation_DataTypes_String_InitQuoted026 [no error or
exception]
ScriptLanguage_Validation_DataTypes_String_InitUnquoted026 [no error or
exception]
ScriptLanguage_Validation_DataTypes_String_MsUnquoted026 [no error or
exception]
ScriptLanguage_Validation_FilePaths_1 [no error or exception]
Other bad test cases
GMT-3371_ElapsedTimeContextBug', ...
ReportFile_Hardware_Thruster_parameters_ImpulsiveBurn <-- Line endings?
Validations all caught
ErrorModel_Validation_SolveFors [#1 failed]
ErrorModel_Validation_Type [#1 failed]
GroundStation_Validation_AddHardware [#1 failed]
GroundStation_Validation_ErrorModels [#1 failed]
GroundStation_Validation_Humidity [#1 failed]
GroundStation_Validation_IonosphereModel [#1 failed]
GroundStation_Validation_Pressure [#1 failed]
GroundStation_Validation_Temperature [#1 failed]
GroundStation_Validation_TroposphereModel [#1 failed]
Receiver_Validation_PrimaryAntenna [#1 failed]
Simulator_Validation_AddData [#1 failed]
StatisticsAcceptFilter_Validation_DataTypes [#1 failed]
StatisticsAcceptFilter_Validation_FileNames [#1 failed]
StatisticsAcceptFilter_Validation_Trackers [#1 failed]
StatisticsRejectFilter_Validation_DataTypes [#1 failed]
StatisticsRejectFilter_Validation_FileNames [#1 failed]
StatisticsRejectFilter_Validation_Trackers [#1 failed]
TrackingFileSet_Validation_AddTrackingConfig [#1 failed]
TrackingFileSet_Validation_DataFilters [#1 failed]
TrackingFileSet_Validation_FileName [#1 failed]
Transmitter_Validation_Frequency [#1 failed]
Transmitter_Validation_PrimaryAntenna [#1 failed]
Suspicious Activity
EphemPropagator_Code500_BackProp_LeapSecond [pos err 71.3356, tol
3e-06]', ...
Bug-GMT-5667 <-- T/F comparitor; looks like a tolerance thing since
script converged
------------------------------------------------------------------------------
_______________________________________________
Gmat-buildtest mailing list
Gmat-buildtest@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gmat-buildtest
