Re: [Flightgear-devel] Rating System Redux (was Re: Flightgear-devel Digest, Vol 61, Issue 12)
On Thu, May 26, 2011 at 8:30 PM, Vivian Meazza wrote: Stuart Thanks for addressing the points that were hammered out over on the IRC channel. I think the modified system could work. Just a few points remain: There is no penalty for including systems, such as an AP, where none existed on the original. There's not an explicit penalty. but I think Hal has addressed this in the notes for the System criteria: Ignore systems not present on the aircraft IRL. If the real aircraft doesn't have a system (e.g. autopilot), the FG model shouldn't have either and if all systems in the real aircraft are modeled then it scores a 5 even if it is a very simple aircraft. I'm not sure how much of a problem this is. If someone chooses not to disable the generic autopilot for a vintage aircraft, it will have no effect on pilots who choose to fly realistically (they simply won't use it). If the system is exposed in the cockpit, then it is covered by the rating for accuracy of cockpit - a KAP140 in the Sopwith Camel would obviously not be worth a 4 or 5 cockpit rating. That is correct - but it doesn't follow from the criteria quoted above. I've added No unrealistic systems to the System-3 rating criteria, with an exception for autopilot. I've also attempted to provide some useful guidance notes. We're talking here about the difference between a 4 or 5 External Model rating, where we're trying to differentiate between a good external model and one that is as realistic as possible. I think we should differentiate between them if possible, but I'm struggling to think up some objective criteria. Photo-realistic? model resolution of 5cm? Perhaps we end up providing subjective criteria, or some additional guidance in this case? I think guidance - livery shouldn't be a criterion for realism, but it might form part of it. Realism is the goal. I've modified the rating and guidance as follows: # 4: Accurate 3D model with animated control surfaces, gear, prop, livery support (if applicable). # 5: Highly accurate 3D model (down to minor components such as control rods), with animated control surfaces, gear, prop, livery support, tyre smoke, shader effects. Objectively differentiating between a 4 and a 5 is very difficult. As a guideline, a 5 model is as realistic as possible given the available rendering technology. I don't have a good answer for the other items. Some are nice-to-haves that enrich the simulation experience but don't impact simulation of flight itself, but others (such as a co-pilot) are more important for multi-crew aircraft. Call them all advanced features. That could be a/the criterion for advanced production I'm not sure. The Advanced production bar is already very high - two 5s and two 4s. I'm not sure if any aircraft will actually gain it! Of course, we're trusting that aircraft developers are going to apply the rating criteria accurately to the best of their ability. Yes - I think perhaps a bit of spot-checking to keep us all honest? Assuming this gets widely adopted, I think it'll be self-policing. Users are going to notice if an aircraft falls below the general criteria. Oh and, finally finally - the model with the highest score might be so good that the framerate means that it can only be used on high-end systems or away from detailed airports. This limitation should be noted somewhere. I don't have a good answer to that. Does that become criteria for a 5 in External Model? I think this ends up back as something subjective. I think we need some form of bench-mark - perhaps the default model at KSFO with certain (all?) features enabled. The aircraft to be rated scores a % framerate above or below this norm? Thinking aloud here a bit. Perhaps that's a bit too fancy. I think that's going to vary so much between graphics systems, plus I'm not sure that graphics degradation is linear - it always seems to fall off a cliff for me :) If enough people rate their aircraft and we can use it to provide a better download page for the upcoming release, it will succeed. Let's hope - some aircraft developers have an awful lot of aircraft to rate. Given his standardized workflow, I think Helijah will be able to apply pretty much the same rating to most of his new models, and retrospectively fit his existing aircraft as well. Hal V. Engel wrote: Hadn't thought about that at all. I've added it to the criteria for a 4 rating. I would treat these as just another system. I think the systems catigory is a difficult one because of how much difference there is between very simple aircraft (think sailplane) and a very complex one (think Concorde). This makes it very difficult to have a rating system that results in similar scores for aircraft that have proportionally complete systems but that are of very different complexity. I am not sure how to improve this but I think it is important to keep it simple. I've moved the
Re: [Flightgear-devel] Rating System Redux (was Re: Flightgear-devel Digest, Vol 61, Issue 12)
On Monday, May 30, 2011 12:47:41 PM Stuart Buchanan wrote: I don't have a good answer for the other items. Some are nice-to-haves that enrich the simulation experience but don't impact simulation of flight itself, but others (such as a co-pilot) are more important for multi-crew aircraft. Call them all advanced features. That could be a/the criterion for advanced production I'm not sure. The Advanced production bar is already very high - two 5s and two 4s. I'm not sure if any aircraft will actually gain it! I would expect that at this point only a few aircraft out there are close to or are advanced production quality. It is a very high standard and any aircraft that is that far along should really stand out. I would expect that most of the most advanced current models only need perhaps 1 or 2 points to get there but adding points when the models are that far along is a lot of work. But I would be surprised if there were more than a handful of aircraft that were far enough along to only need 1 or 2 points to become advanced production. I think I agree with Stuart that having some things called advanced features does not add much if anything to the system particularly when we have so many models that are missing many basic things. An example of one that is close but needs more work is the p51d-jsbsim model. It only needs to improve the external model (add livery support to go from a 3 to a 4) to get to production status and then add one more point in cockpit, external model or systems would make it advanced production. Currently it has the following ratings: rating FDM type=int5/FDM systems type=int4/systems model type=int3/model cockpit type=int4/cockpit /rating The 3D modeling stuff is not my strong suit but I do have new more accurate 3D models for the fuselage and wing (including flaps and aileraons) for the P-51D that I created a while back. I have also more accurately modeled the cooling inlet passages and the oil and coolant radiators so that these will look correct (once textured) when looking into the cooling inlet. I need to uvmap all of this stuff now and this is where I get stuck as I can't figure out how to do this so that the resulting uvmaps can be used to create livery support. Having a nice user friendly uvmap for the fugelage and wings is more or less nessary to move ahead with libery support I think. For Systems adding emergency gear release support, oxygen system support, full cooling system support, VHF radio support, rear warning radar support, IFF support and some missing electrical system stuff would increase this to a 5. The 3D models for the controls for all of these systems are already in the cockpit. One comment about systems. For the P-51 series there are two cooling doors that are used to control cooling airflow. One for the engine coolant and one for the oil cooler. JSBSim has support for the coolant door controls but not for the oil cooler door controls. I have the automatic coolant door stuff modeled but not the automatic oil cooler stuff because of this. I also need to add manual overides for these at some point (the controls are in the cockpit but currently only allow for automatic control). What I am getting at is that some systems can not be fully modeled because of limitations in the FDM being used and aircraft authors should rate these as complete systems if they have modeled everything that is possible with the existing FDM support. For Cockpit adding the fuselage fuel tank and guage, a few missing placards, the arm rest, the map bag and improved texturing would pretty much get it a 5. For some aircraft it may never be possible to get the FDM rating high enough to get more than a 2 or 3 simply because the data needed to do that is not available. These aircraft will never be able to get beyond an early production status unless the author finds a source for the needed information. Hal -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Rating System Redux (was Re: Flightgear-devel Digest, Vol 61, Issue 12)
Stuart Thanks for addressing the points that were hammered out over on the IRC channel. I think the modified system could work. Just a few points remain: There is no penalty for including systems, such as an AP, where none existed on the original. There's not an explicit penalty. but I think Hal has addressed this in the notes for the System criteria: Ignore systems not present on the aircraft IRL. If the real aircraft doesn't have a system (e.g. autopilot), the FG model shouldn't have either and if all systems in the real aircraft are modeled then it scores a 5 even if it is a very simple aircraft. I'm not sure how much of a problem this is. If someone chooses not to disable the generic autopilot for a vintage aircraft, it will have no effect on pilots who choose to fly realistically (they simply won't use it). If the system is exposed in the cockpit, then it is covered by the rating for accuracy of cockpit - a KAP140 in the Sopwith Camel would obviously not be worth a 4 or 5 cockpit rating. That is correct - but it doesn't follow from the criteria quoted above. I don't think it's unreasonable for vintage aircraft to have access to a radio, for example. A modern pilot flying a vintage aircraft would carry a hand-held. Yes - it depends on whether we are modeling the original, or a currently flying example. I've never quite made up my mind on that one. The use of shaders etc. may or may not enhance the realism of the model and in some cases could be used inappropriately. This is a subjective assessment, and perhaps could be removed from the points system. Livery support is not necessarily an enhancement - it is not appropriate for all models. We're talking here about the difference between a 4 or 5 External Model rating, where we're trying to differentiate between a good external model and one that is as realistic as possible. I think we should differentiate between them if possible, but I'm struggling to think up some objective criteria. Photo-realistic? model resolution of 5cm? Perhaps we end up providing subjective criteria, or some additional guidance in this case? I think guidance - livery shouldn't be a criterion for realism, but it might form part of it. Realism is the goal. I'm not clear if you are awarding points for underwing stores and the like. Hadn't thought about that at all. I've added it to the criteria for a 4 rating. We have additional features such as co-pilot/RIO over MP, Wingmen, Formation Control, Tutorials, Aircraft Specific Help, Contrails, Vapour Trails, and there are probably some I missed. Contrails Vapour trails should probably be covered by the external model, I think. I could add them (along with tyre smoke) as criteria for a Model 5 rating? Yes - tyre smoke is a generic facility - there is no reason for it not being added to a model. I don't have a good answer for the other items. Some are nice-to-haves that enrich the simulation experience but don't impact simulation of flight itself, but others (such as a co-pilot) are more important for multi-crew aircraft. Call them all advanced features. That could be a/the criterion for advanced production And finally - the points system could award a high status to a poor model - there are no points awarded for the accuracy or the fidelity of the 3d model. E.G there is at least one model with afterburners modelled where none existed. I've updated the external model to include the world Accurate for ratings 3-5. Good Of course, we're trusting that aircraft developers are going to apply the rating criteria accurately to the best of their ability. Yes - I think perhaps a bit of spot-checking to keep us all honest? Oh and, finally finally - the model with the highest score might be so good that the framerate means that it can only be used on high-end systems or away from detailed airports. This limitation should be noted somewhere. I don't have a good answer to that. Does that become criteria for a 5 in External Model? I think this ends up back as something subjective. I think we need some form of bench-mark - perhaps the default model at KSFO with certain (all?) features enabled. The aircraft to be rated scores a % framerate above or below this norm? Thinking aloud here a bit. Perhaps that's a bit too fancy. Let's hope that this tool can help to bring some order out of the current chaos. We can but try. Certainly this seems to have a bit more momentum behind it than previous attempts, based on the feedback here and on IRC. If enough people rate their aircraft and we can use it to provide a better download page for the upcoming release, it will succeed. Let's hope - some aircraft developers have an awful lot of aircraft to rate. Vivian -- vRanger cuts backup time in half-while increasing
Re: [Flightgear-devel] Rating System Redux (was Re: Flightgear-devel Digest, Vol 61, Issue 12)
On Thursday, May 26, 2011 06:31:13 AM Stuart Buchanan wrote: On Thu, May 26, 2011 at 9:45 AM, Vivian Meazz awrote: Thanks for addressing the points that were hammered out over on the IRC channel. I think the modified system could work. Just a few points remain: There is no penalty for including systems, such as an AP, where none existed on the original. There's not an explicit penalty. but I think Hal has addressed this in the notes for the System criteria: Ignore systems not present on the aircraft IRL. If the real aircraft doesn't have a system (e.g. autopilot), the FG model shouldn't have either and if all systems in the real aircraft are modeled then it scores a 5 even if it is a very simple aircraft. I'm not sure how much of a problem this is. If someone chooses not to disable the generic autopilot for a vintage aircraft, it will have no effect on pilots who choose to fly realistically (they simply won't use it). On the p51d-jsbsim I have added a tuned autopilot but it is only available by using the menu system since the real thing (IE. my model is as it would have been in 1945) would not have one. But it is VERY useful for test flights so it was worth the effort to create it. I don't think this should result in a reduced Systems score unless it is exposed in the cockpit. So I agree with Stuart. If the system is exposed in the cockpit, then it is covered by the rating for accuracy of cockpit - a KAP140 in the Sopwith Camel would obviously not be worth a 4 or 5 cockpit rating. I don't think it's unreasonable for vintage aircraft to have access to a radio, for example. A modern pilot flying a vintage aircraft would carry a hand-held. I agree with this and as others have pointed out it depends on what you are modeling - IE. how the aircraft was back in the day or how it might be used today. These are really two different aircraft or at least two differenet configurations. The use of shaders etc. may or may not enhance the realism of the model and in some cases could be used inappropriately. This is a subjective assessment, and perhaps could be removed from the points system. Livery support is not necessarily an enhancement - it is not appropriate for all models. We're talking here about the difference between a 4 or 5 External Model rating, where we're trying to differentiate between a good external model and one that is as realistic as possible. I think we should differentiate between them if possible, but I'm struggling to think up some objective criteria. Photo-realistic? model resolution of 5cm? Setting up for liveries appears to be a significant non-trivial task although I have not looked into it in detail. If the model is intended to be of a specific aircraft as it existed at a particualr point in time then liveries make no sense for that model. On the other hand a particular aircraft may have a long history and using liveries would make it possible to model the same aircraft at different points in it's history. Perhaps we end up providing subjective criteria, or some additional guidance in this case? I'm not clear if you are awarding points for underwing stores and the like. Hadn't thought about that at all. I've added it to the criteria for a 4 rating. I would treat these as just another system. I think the systems catigory is a difficult one because of how much difference there is between very simple aircraft (think sailplane) and a very complex one (think Concorde). This makes it very difficult to have a rating system that results in similar scores for aircraft that have proportionally complete systems but that are of very different complexity. I am not sure how to improve this but I think it is important to keep it simple. We have additional features such as co-pilot/RIO over MP, Wingmen, Formation Control, Tutorials, Aircraft Specific Help, Contrails, Vapour Trails, and there are probably some I missed. Contrails Vapour trails should probably be covered by the external model, I think. I could add them (along with tyre smoke) as criteria for a Model 5 rating? I don't have a good answer for the other items. Some are nice-to-haves that enrich the simulation experience but don't impact simulation of flight itself, but others (such as a co-pilot) are more important for multi-crew aircraft. And finally - the points system could award a high status to a poor model - there are no points awarded for the accuracy or the fidelity of the 3d model. E.G there is at least one model with afterburners modelled where none existed. I've updated the external model to include the world Accurate for ratings 3-5. Of course, we're trusting that aircraft developers are going to apply the rating criteria accurately to the best of their ability. Oh and, finally finally - the model with the highest score might be so good that the framerate means that it
