Hi Pasi, On Feb 23, 2018 08:05, "Pasi Keränen" <pasi.kera...@qt.io> wrote:
Hi there, +1 for having a generic telemetry plugin in Qt. I planned to stay out of this, but -1 since I am totally confused about the scope of this project at this point. So are we talking about a generic telemetry framework for Qt applications or about watching Qt Creator users specifically? Tino started by making the claim that this is a generic library and then kept listing Qt Creator specific integrations. You focus entirely onto the generic part, leaving out creator completely. What exactly are we talking about here? A Qt creator spyware plug-in (which does more than what we discuss here I hope:-) would be a matter of a couple of weeks to do, putting a generic framework into place with all the bits and pieces for that to be actually useful in a wide list of possible contexts is a very different beast. This is great initiative and very much the way today's app and application industry works. UX studies performed by UX experts have been minimized and targeted for specific (usually new/experimental) features or upcoming new software (like we did with Qt 3D Studio back in last spring). And the mass information on "how do our users use the SW? do they find the stuff we've put in there? how often they hit a wall in doing sequence X? how many crashes do they experience when doing Y?" is collected via automated telemetry. It is great as it brings data from the actual user in their actual work and you can then use that to concentrate on functionality that really matters to your users. Making stuff they repeat hundred times a week easier and faster, making them more productive. I see value in this approach when you can do lots of small releases fast. So you can do evaluate the effect of small changes by doing one change to evaluate per release and measure how that effects usage. We can not do more than two releases per year in Qt. Is this approach even applicable to us? I want to also point out that answering any of the questions you used as an example require *way* more information than I am even remotely comfortable to collect. I see definite need for this in Qt 3D Studio so please don’t make this just with Qt Creator in mind. Also, in my humble opinion, in order to be relevant in today's UI development, Qt should also offer this kind of a plug-in to our customers. A ready-to-go plug-in that automatically ensures the data is collected in a way that fulfills data privacy acts and respects the privacy of the user would be great. Especially for startups and smaller companies, but also for bigger companies wanting to switch to the modern way of doing UI development. It is not as easy to do as one might think at glance. I agree that having a general framework has value to some customers. Such a framework would need to be integrated into big solutions like Google analytics or something similar so the users can actually evaluate the collected data and cross-reference that to other data they collect. Just dumping data into some server somewhere does not help anybody. Will the server-side code be part of this project by the way? What about evaluation of the collected data? Is that in scope for this project, too? I would ask anyone who has not done work with usability and user experience people in the past to give this way of working a chance. I've worked 7 years in application development while we grew usability knowledge in the team over that time. The first time I got to observe a real world user working with our software in actual real world situation was eye opening. We had gotten so many important things wrong in our idealistic thinking and forgotten to handle certain cases that occur on the field. Also you become blind to your own creations faults as you just know how the software works. It's just a fact of life. Sure. We do way too little validation of what we do against the real world usage. Let us do some usability studies, let us talk to users, let us do surveys, let's evaluate all the data users provide to us on the mailing list, the bug tracker, the forums, stack overflow and in lots of other places. We do have a very active and supportive community of users and customers, let us use the feedback they provide already! Collecting some semi-random data from a self-selected group of users and dumping that onto a server somewhere is next to useless compared to all the other options we have already. Even in an ideal situation (which we do not have within Qt!), metrics are not comparable to a real user study where you actually watch users doing their thing. This is even more true when not leaving out any information that can be used to identify individual users from the collected data, which we obviously will do. Best Regards, Tobias
_______________________________________________ Qt-creator mailing list Qt-creator@qt-project.org http://lists.qt-project.org/mailman/listinfo/qt-creator