Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
And here is one company, which can do the mind reading for you (as a side project they can find out what your test subjects think about Saddam, W. Bush, their reaction to pain etc.): http://ahe6.tripod.com/cognitive.eng/id45.html On Fri, Apr 25, 2008 at 2:09 PM, Oleh Kovalchuke <[EMAIL PROTECTED]> wrote: > On Tue, Apr 22, 2008 at 7:23 AM, Todd Zaki Warfel <[EMAIL PROTECTED]> > wrote: > > > > Oh, oh, oh, I know—what we need is eye tracking with mind reading. > > Now, that's useful. > > > > > This is not too far fetched. > > Jared is right: there is no 100% correlation between eye fixation and > locus of attention (or understanding what is being viewed). That can be > solved soon enough: > > "The scientists used a functional magnetic resonance imaging machine -- a > real-time brain scanner -- to record the mental activity of a person looking > at thousands of random pictures: people, animals, landscapes, objects, the > stuff of everyday visual life. With those recordings the researchers built a > computational model for predicting the mental patterns elicited by looking > at any other photograph. When tested with neurological readouts generated by > a different set of pictures, the decoder passed with flying colors, > identifying the images seen with unprecedented accuracy. " > http://www.wired.com/science/discoveries/news/2008/03/mri_vision > > I assume the image of shopping cart would elicit different pattern from > the image of login field or the ad banner, since they have different meaning > to the website visitor. > > -- > Oleh Kovalchuke > Interaction Design is design of time > http://www.tangospring.com/IxDtopicWhatIsInteractionDesign.htm > Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Tue, Apr 22, 2008 at 7:23 AM, Todd Zaki Warfel <[EMAIL PROTECTED]> wrote: > Oh, oh, oh, I know—what we need is eye tracking with mind reading. > Now, that's useful. > This is not too far fetched. Jared is right: there is no 100% correlation between eye fixation and locus of attention (or understanding what is being viewed). That can be solved soon enough: "The scientists used a functional magnetic resonance imaging machine -- a real-time brain scanner -- to record the mental activity of a person looking at thousands of random pictures: people, animals, landscapes, objects, the stuff of everyday visual life. With those recordings the researchers built a computational model for predicting the mental patterns elicited by looking at any other photograph. When tested with neurological readouts generated by a different set of pictures, the decoder passed with flying colors, identifying the images seen with unprecedented accuracy. " http://www.wired.com/science/discoveries/news/2008/03/mri_vision I assume the image of shopping cart would elicit different pattern from the image of login field or the ad banner, since they have different meaning to the website visitor. -- Oleh Kovalchuke Interaction Design is design of time http://www.tangospring.com/IxDtopicWhatIsInteractionDesign.htm Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Andre, In all of the discussion, the initial question got unanswered: * Is eyetracking too expensive? * "Expensive" is all relative. Currently, you have three options: 1) outsourced data collection and analysis: you can have someone expert in eyetracking collect and analyze your data for you. This has the benefit that it solves the problem many people brought up which is that experience counts and it's harder to analyze eyetracking data than most people realize. Eyetools, Inc. (www.eyetools.com) does this for a lot of clients and agencies (full disclosure: my company -- I'm the founder and CEO). There are other companies as well (actually, quite a few usability companies have started offering eyetracking in the last year). Cost: $2,800 - $5,000 including recruiting, incentives, and facilities for testing a homepage/landing page/etc. (I'm not going to go into details of pricing since this is probably not the appropriate place for that.) 2) You can buy your own equipment: there are a number of hardware manufactures out there with pricing ranging from $20k-$44k for the hardware and software necessary to run web usability tests. Hardware manufacturers include LC Technologies (www.eyegaze.com), Eyetech Digital Systems (www.eyetechds.com), SMI (www.smi.com), ASL (www.a-s-l.com, though recently they've been doing less web usability stuff), and SmartEye (www.smarteye.se) are some that come to the top of my mind. (full disclosure -- Eyetools is also a Tobii reseller, though you can buy direct from Tobii also). Cost: $44,000+ for a typical usability set up including hardware and software necessary for doing web testing 3) You can rent a system: this raises the larger issue of "will you know how to get valid data and analyze it correctly" which no matter what I say no one will believe since I'm in the business of providing services. However, experience does count, as you can well imagine. There have been some resources appearing on the web to support people figuring it out on their own, though it's not clear that they'll actually truly teach you all you need to know, so again, you're on your own. This all goes to the second part of the question which is "is it complicated" and one can look at this discussion thread to decide for themselves. Cost: around $7,400 for renting for two months, running 4 studies (prices vary depending on how many studies you run -- the more you run, the more you pay). Best regards, -Greg Greg Edwards CEO & Founder, Eyetools Inc. [EMAIL PROTECTED] 916.792.4538 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
As others have already pointed out, the primary benefactor of eye-tracking studies is the coordinator of the studies. After many such studies, I gradually have absorbed what textbooks could not teach me effectively about this science. What draws the human eye is not quite unpredictable, but the wide variance does not lend itself to easy rule-making. Madison Avenue has been using this technology for years for somewhat nefarious purposes. We need to embrace it and put the knowledge to good use across the discipline of interaction and experience design. How could we tolerate the idea that it would be okay for marketing and advertising folks to know more about this area than ourselves? Here's the rub though: I have always found it challenging to use the eye-tracking data to correct my own designs. When I tried to do this initially, I would never get it right: I would re-submit "corrected" designs for new eye-tracking and I'd get even worse results. Persistence was key. Eye-tracking studies have made me a better designer. And my clients love it. Dave -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Larry Tesler Sent: Tuesday, April 22, 2008 11:22 PM To: Jared M. Spool Cc: [EMAIL PROTECTED] Subject: Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated? Jared, If most readers think this has gone on too long, we should wind it down. On Apr 22, 2008, at 5:59 AM, Jared M. Spool wrote: > > On Apr 22, 2008, at 3:08 AM, Larry Tesler wrote: > >> The fact that different observers see different things in the same >> raw eye tracking data is of no more concern to me than the fact >> that different players count a different number of words on the >> same Boggle board. Some people see words that are hidden in plain >> sight; some do not. But noticed or not, the words are there. In the >> tea leaves, there are no hidden words. > > Larry, > > I have no doubt that the observations are of interest. > > My point is that the inferences drawn from those observations have > little-to-no validity, thus the tea leaf analogy. I consider them valid if they inspire us to make design changes that lead to improvements in objective metrics. In any study, with or without an eye tracker, I look for: - a well-designed experiment - clean data - appropriate, error-free analysis - perceptive observation (which may require several observers to point things out to each other and reach consensus, as radiologists often do when faced with difficult images) - generation of hypotheses consistent with the study observations and any other available observations - prioritization of those hypotheses - generation of design solutions that respond to the most likely hypothesis - implementation - bucket testing - if no improvement is seen, iterate with alternate hypotheses > If someone fixates on a link for a unusually large time, does that > mean they are confused by it? Or they aren't confused, but are > trying to decide if its what they want? Or they know whether they > want it or not but are considering something else? If the user's mental state matters to you, ask the user what it was. They may know. If they do not know, devise a more clever experiment. But sometimes, the user's mental state doesn't matter. We may have run the test because too few people were clicking on the link. We thought perhaps they didn't even look at the area of the page that contained the link. The tracker has refuted our hypothesis. We know that some people look straight at the link and still do not click it. Other data may be needed if we want to find out why. But the study was a success. It achieved its goal. > Different inferences will lead to completely different design > solutions. Are you saying it doesn't matter which inference (and > therefore, which design solution) the observers choose? If the radiologists call your malignant tumor benign, or vice versa, you may receive the wrong treatment, which could be a costly mistake. But design changes that eye tracking studies inspire often entail simple modifications to layout, color, size, typeface, etc., that help to steer attention. They are often cheap to implement. If there are two competing inferences, you can often try both implied solutions. Of course, if you had both designs in mind (and particularly the one that ultimately proved to be best) before you ran the eye tracking study, then the study was a waste of time. If that is the situation you are in, and you never want to run a study that may simply confirm that you were right, then your point is valid. But it is not always the situation. There may be no unrefuted theory about why users are not clicking the link. There may be no considered designs that would increase clicks. There may b
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Jared, If most readers think this has gone on too long, we should wind it down. On Apr 22, 2008, at 5:59 AM, Jared M. Spool wrote: > > On Apr 22, 2008, at 3:08 AM, Larry Tesler wrote: > >> The fact that different observers see different things in the same >> raw eye tracking data is of no more concern to me than the fact >> that different players count a different number of words on the >> same Boggle board. Some people see words that are hidden in plain >> sight; some do not. But noticed or not, the words are there. In the >> tea leaves, there are no hidden words. > > Larry, > > I have no doubt that the observations are of interest. > > My point is that the inferences drawn from those observations have > little-to-no validity, thus the tea leaf analogy. I consider them valid if they inspire us to make design changes that lead to improvements in objective metrics. In any study, with or without an eye tracker, I look for: - a well-designed experiment - clean data - appropriate, error-free analysis - perceptive observation (which may require several observers to point things out to each other and reach consensus, as radiologists often do when faced with difficult images) - generation of hypotheses consistent with the study observations and any other available observations - prioritization of those hypotheses - generation of design solutions that respond to the most likely hypothesis - implementation - bucket testing - if no improvement is seen, iterate with alternate hypotheses > If someone fixates on a link for a unusually large time, does that > mean they are confused by it? Or they aren't confused, but are > trying to decide if its what they want? Or they know whether they > want it or not but are considering something else? If the user's mental state matters to you, ask the user what it was. They may know. If they do not know, devise a more clever experiment. But sometimes, the user's mental state doesn't matter. We may have run the test because too few people were clicking on the link. We thought perhaps they didn't even look at the area of the page that contained the link. The tracker has refuted our hypothesis. We know that some people look straight at the link and still do not click it. Other data may be needed if we want to find out why. But the study was a success. It achieved its goal. > Different inferences will lead to completely different design > solutions. Are you saying it doesn't matter which inference (and > therefore, which design solution) the observers choose? If the radiologists call your malignant tumor benign, or vice versa, you may receive the wrong treatment, which could be a costly mistake. But design changes that eye tracking studies inspire often entail simple modifications to layout, color, size, typeface, etc., that help to steer attention. They are often cheap to implement. If there are two competing inferences, you can often try both implied solutions. Of course, if you had both designs in mind (and particularly the one that ultimately proved to be best) before you ran the eye tracking study, then the study was a waste of time. If that is the situation you are in, and you never want to run a study that may simply confirm that you were right, then your point is valid. But it is not always the situation. There may be no unrefuted theory about why users are not clicking the link. There may be no considered designs that would increase clicks. There may be too many credible designs--more than one has the time and staff to implement and test. Or you may simply want to confirm other data or hunches. > When you back an eye-tracking supporter into a corner about this, > they all say, "Well, you should only use eye tracking in conjunction > with other data collection tools and techniques to verify your > inferences." In almost all cases, the "other data collection tools > and techniques" would yield just as much value without the eye > tracking as with it, so what's the benefit? The eye tracking test may have been the source of the first clue as to what ails your interface. Without it, you may never have thought to try those other tools and techniques. Or the eye tracking test may help to rule out hypotheses that one has generated from the use of other tools and techniques. > Second, in almost all uses of eye tracking I've seen in the last 5 > years, it's in the form of twisting the meaning of the heatmap/plot > diagram/tea leaf reading into supporting whatever wacky inference > the specialist wants to support. "See that big red spot there. That > means the users are confused" v. "See that big red spot there, that > means we fixed the design." I agree that features of the heat map don't tell you mental states. But if they are inconsistent with hypotheses about mental states, they may call those hypotheses into question. "See that big red spot over there. Maybe they
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Todd wrote: > Oh, oh, oh, I know%u2014what we need is eye tracking with mind reading. Now, that's useful. Hah! If we had mind reading, we wouldn't need the eye tracker. Jared Jared M. Spool User Interface Engineering 510 Turnpike St., Suite 102, North Andover, MA 01845 e: [EMAIL PROTECTED] p: 1 978 327 5561 http://uie.com Blog: http://uie.com/brainsparks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=28208 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Apr 22, 2008, at 8:59 AM, Jared M. Spool wrote: > If someone fixates on a link for a unusually large time, does that > mean they are confused by it? Or they aren't confused, but are > trying to decide if its what they want? Or they know whether they > want it or not but are considering something else? Oh, oh, oh, I know—what we need is eye tracking with mind reading. Now, that's useful. Cheers! Todd Zaki Warfel President, Design Researcher Messagefirst | Designing Information. Beautifully. -- Contact Info Voice: (215) 825-7423 Email: [EMAIL PROTECTED] AIM:[EMAIL PROTECTED] Blog: http://toddwarfel.com Twitter:zakiwarfel -- In theory, theory and practice are the same. In practice, they are not. Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Apr 22, 2008, at 3:08 AM, Larry Tesler wrote: > The fact that different observers see different things in the same > raw eye tracking data is of no more concern to me than the fact that > different players count a different number of words on the same > Boggle board. Some people see words that are hidden in plain sight; > some do not. But noticed or not, the words are there. In the tea > leaves, there are no hidden words. Larry, I have no doubt that the observations are of interest. My point is that the inferences drawn from those observations have little-to-no validity, thus the tea leaf analogy. If someone fixates on a link for a unusually large time, does that mean they are confused by it? Or they aren't confused, but are trying to decide if its what they want? Or they know whether they want it or not but are considering something else? Different inferences will lead to completely different design solutions. Are you saying it doesn't matter which inference (and therefore, which design solution) the observers choose? When you back an eye-tracking supporter into a corner about this, they all say, "Well, you should only use eye tracking in conjunction with other data collection tools and techniques to verify your inferences." In almost all cases, the "other data collection tools and techniques" would yield just as much value without the eye tracking as with it, so what's the benefit? Second, in almost all uses of eye tracking I've seen in the last 5 years, it's in the form of twisting the meaning of the heatmap/plot diagram/tea leaf reading into supporting whatever wacky inference the specialist wants to support. "See that big red spot there. That means the users are confused" v. "See that big red spot there, that means we fixed the design." If there really is something to this eye tracking thing, I'd think you'd want your team members to all look at the same heat map and come to somewhat similar design implications. Eyetracking equipment: $30,000 Ouija Board: $5 Quality design based on solid inferences from rich, meaningful data: Priceless That's my take. Jared Jared M. Spool User Interface Engineering 510 Turnpike St., Suite 102, North Andover, MA 01845 e: [EMAIL PROTECTED] p: +1 978 327 5561 http://uie.com Blog: http://uie.com/brainsparks Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Jared, Eye tracking is of value when someone sees something in the data that leads to a new and valuable insight. The fact that different observers see different things in the same raw eye tracking data is of no more concern to me than the fact that different players count a different number of words on the same Boggle board. Some people see words that are hidden in plain sight; some do not. But noticed or not, the words are there. In the tea leaves, there are no hidden words. Adding observers to any team that is interpreting raw data makes it more likely that someone will notice something subtle, but important, that other observers have missed. This is as true in design research as it is in radiology, geophysicists or espionage. Larry Tesler On Apr 19, 2008, at 6:46 PM, Jared M.Spool wrote: > ... Deducing information > about a design from eyetracking is equivalent to reading tea leaves > and using a ouija board. > > The latter are cheaper, but just as reliable. > > Every person I know who swears by eyetracking and has stories on how > its helped them can't explain how they would've gotten the same > results if some other professional had looked at the same raw data. > Until we can get to that point, the reader of the data will be more > important than the data itself, thereby making tea leaf reading a > viable alternative. Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
[IxDA Discuss] Is Eye Tracking too expensive or complicated?
In the end, it's on what people click, which is really important. And even more important is the information people are really looking for and the findability of that information. It all comes down to offering the right information content in a nice way. Without the right information, your site may be optimized the way you want, people won't be happy. It's a bit back to basics, but basics are extremely important. Pieter Jansegers http://webosophy.ning.com Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Apr 20, 2008, at 3:54 PM, Christopher Fahey wrote: > Nor can they explain why they wouldn't get *better* results and > *better* recommendations from simply showing the UI to a half-decent > user interface designer for 20 minutes Eye-tracking should not be used on its own—if used at all, it should be for supplementary input. Yes, it tells you what someone is looking at, but doesn't tell you why—and the "why" is critical to finding the appropriate design solution. Cheers! Todd Zaki Warfel President, Design Researcher Messagefirst | Designing Information. Beautifully. -- Contact Info Voice: (215) 825-7423 Email: [EMAIL PROTECTED] AIM:[EMAIL PROTECTED] Blog: http://toddwarfel.com Twitter:zakiwarfel -- In theory, theory and practice are the same. In practice, they are not. Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Sounds like what's missing here is a set of consistent, objective and reliable guidelines for interpreting eye-tracking data (and potentially usability findings in general). For example a fixation of an a priori specified minimum duration on a link in conjunction with a user failing to click the link AND the user reporting that the link was seen would strongly indicate that indeed it was seen : ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=28208 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Sun, Apr 20, 2008 at 11:43 PM, Jared M. Spool <[EMAIL PROTECTED]> wrote: > All you know is that the eye tracker registered that they fixated on the > link and that they didn't click. > The notion that they didn't understand the link is one inference. > It's not the only inference. It may not be the right inference. > It is purely *your* interpretation that the user didn't understand it. > One way of double-checking the inferences is to ask the participant. What I've observed in eyetracking has been confirmed by participants enough that I know that the premises of eyetracking are true. Seems like you have had a different experience, so I'll be curious to hear what people have to say at UPA about it. (And you could've gotten there without the eye tracking data.) > This assumes that you knew there was an issue to begin with, or that the type of study allows you to follow up. Neither is always the case. In fact, in psychographic phenomena, it's pretty amazing what people can see > and deduce from the peripheral vision. There's a lot happening within 140 > degrees of the focal point. > And it's pretty amazing what is lost within the center gaze area, > especially with people who have field issues that are frequent in males over > 40, females over 50, and anyone suffering from optic neuritis or other > immune-deficiency-based symptoms. (In MS patients, for example, optic > neuritis frequently shows up in late teens, early 20s.) Sounds interesting, do you have a link? Is the device all they need to make the judgments necessary to provide good > design advice? Of course not. In nowhere here have I said that eyetracking was the only way to make judgments. It's just another tool. IMO, the main problems with eyetracking are 1) the multiple participant data (heatmaps) doesn't always make sense, 2) it is time consuming to use, and 3) the initial cost of the equipment is ridiculously high for the benefit that you get. It is not that the premises are wrong. On this point I think that we disagree, so let's just leave it there. Paul Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Apr 20, 2008, at 7:45 PM, Paul Nuschke wrote: > Imagine that a user needs to click on a link to go somewhere. If she > fixates > on the link and don't click it, then that's pretty good evidence > that she > did not understand the link. All you know is that the eye tracker registered that they fixated on the link and that they didn't click. The notion that they didn't understand the link is one inference. It's not the only inference. It may not be the right inference. It is purely *your* interpretation that the user didn't understand it. (And you could've gotten there without the eye tracking data.) >> We know that people see things through their peripheral vision, such >> as the scroll bar, so that's not recorded by the eye tracker. That >> means we can't even assume that when someone doesn't gaze at a spot >> that it wasn't seen. > > True, but that's a good thing. You can't read or see fine details in > your > peripheral vision, so even if you notice something it doesn't mean > that you > looked at it enough to understand what it contained (unless the > important > details were very big). Again. Your inference. You don't have any evidence to actually know that's true. In fact, in psychographic phenomena, it's pretty amazing what people can see and deduce from the peripheral vision. There's a lot happening within 140 degrees of the focal point. And it's pretty amazing what is lost within the center gaze area, especially with people who have field issues that are frequent in males over 40, females over 50, and anyone suffering from optic neuritis or other immune-deficiency-based symptoms. (In MS patients, for example, optic neuritis frequently shows up in late teens, early 20s.) So, you are just inferring meaning to the data you're collecting. > In the example above, even if the user noticed that > a link existed, if she did not attend to it, then she would not have > been > able to read it. Your inference. There are other likely inferences too. >> Show me a study that shows that N separate evaluators looked at the >> same eye tracking data and came away with the same conclusions and >> I'll change my mind. > > That some data does not make sense is not a phenomenon unique to > eyetracking. I've seen plenty of different interpretations of > statistics as > well. Ok. Does that make eyetracking work? Not buying it. Still think it's up to the interpreter of the eye tracker. Let me put it another way: Would you, Paul, be comfortable letting your clients to use the eye tracker without any help in interpreting data from you. Is the device all they need to make the judgments necessary to provide good design advice? Jared Jared M. Spool User Interface Engineering 510 Turnpike St., Suite 102, North Andover, MA 01845 e: [EMAIL PROTECTED] p: +1 978 327 5561 http://uie.com Blog: http://uie.com/brainsparks Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
This may be obvious or trivial, but it was a new insight to me. I've never used eye tracking in software/web design or thought it would have much utility. Nonetheless I wandered up to one of the eye-tracking vendors at the CHI conference and got into a conversation with the rep. He said that most of their sales and emphasis were on contexts where there was something other than a single screen to look at -- such as automobile dashboards, complex control panels like nuclear power plants, and the like. In those situations, seeing where people are looking in response to stimuli like alerts, gauges, oncoming obstacles etc., that can come from many different directions, is very important and the eye tracking apparatus can be extremely helpful. That made a lot of sense to me. Al Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Sun, Apr 20, 2008 at 5:50 PM, Jared M. Spool <[EMAIL PROTECTED]> wrote: > Paul Nuschke wrote: > > Eyetracking lets you see where people are looking in real time. > > Yes. But just because you know where someone looks or doesn't look > doesn't mean you know anything about what they see, what they wanted > to see, and what they didn't see. It's not clear to me how one > interprets the "they gazed at this point on the screen for 400 ms" > information. Was that good? Was that bad? Imagine that a user needs to click on a link to go somewhere. If she fixates on the link and don't click it, then that's pretty good evidence that she did not understand the link. We know that people see things through their peripheral vision, such > as the scroll bar, so that's not recorded by the eye tracker. That > means we can't even assume that when someone doesn't gaze at a spot > that it wasn't seen. True, but that's a good thing. You can't read or see fine details in your peripheral vision, so even if you notice something it doesn't mean that you looked at it enough to understand what it contained (unless the important details were very big). In the example above, even if the user noticed that a link existed, if she did not attend to it, then she would not have been able to read it. Show me a study that shows that N separate evaluators looked at the > same eye tracking data and came away with the same conclusions and > I'll change my mind. That some data does not make sense is not a phenomenon unique to eyetracking. I've seen plenty of different interpretations of statistics as well. > >One analogy I find useful, in terms of understanding what the > participant is doing/thinking, is that having eyetracking versus not > having eyetracking is like testing in person versus testing remotely. > > You lost me there. > In remote testing, you loose voice quality and you don't see mannerisms, facial expressions, etc. In "in person" testing, you have gestures and facial expressions, and voice inflections. In eyetracking, you add the ability to see where they are looking. You lose something too, though, in your testing methodology, but that's another e-mail thread. Paul Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Apr 20, 2008, at 6:18 PM, mark schraad wrote: > I am sure there is a technical term for > this 'attention periphery' but I have not found it in the research > yet. Search for "situation inattentional blindness". The primary work was done by Simons at U of Illinois at Urbana-Champagne. Jared Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
I have been observing (pardon the pun) how often people look right at something and don;t see it. I am sure there is a technical term for this 'attention periphery' but I have not found it in the research yet. I would love to see the results and analysis of an eye tracking expert of subject watching the now classic dancing bear in the basketball game. Mark http://www.dothetest.co.uk/ On Apr 20, 2008, at 4:56 PM, Paul Nuschke wrote: > Eyetracking lets you see where people are looking in real time. Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
I'm wondering about the need of eye tracking on any particular page. I mean : it's nice to repeat scientific tests over and over again to control the results. But I don't think it could add specific value to website analysis as such to do it on every site over and over again. I don't see why the results would differ from a general analysis of the website based on the principles obtained in research previously. But, hey, if you can convince a client more easily to put his/her money into your pocktets showing a nice reddish glow on a graph... who I am to stop you ? Pieter Jansegers http://webosophy.ning.com . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=28208 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Paul Nuschke wrote: > Eyetracking lets you see where people are looking in real time. Yes. But just because you know where someone looks or doesn't look doesn't mean you know anything about what they see, what they wanted to see, and what they didn't see. It's not clear to me how one interprets the "they gazed at this point on the screen for 400 ms" information. Was that good? Was that bad? We know that people see things through their peripheral vision, such as the scroll bar, so that's not recorded by the eye tracker. That means we can't even assume that when someone doesn't gaze at a spot that it wasn't seen. With eye trackers, we have a bunch of observations but no way to determine the proper inferences. Instead, all of the value of an eye tracker comes from the interpretation. Show me a study that shows that N separate evaluators looked at the same eye tracking data and came away with the same conclusions and I'll change my mind. Until then, I'll continue to group it with tarot cards and palm reading as a fine art. > Without considering post-test analysis, this has real value in helping the facilitator better understand what is happening without interfering. Exactly my point. As the President of Best Buy, John "JT" Thompson, once told me (while I was delivering a great presentation with a ton of data): "I worked for Jack Welch at GE for 17 years and if I learned anything while I was there, it was this: If you torture data long and hard enough, it will confess to anything you want." >One analogy I find useful, in terms of understanding what the participant is doing/thinking, is that having eyetracking versus not having eyetracking is like testing in person versus testing remotely. You lost me there. > I wonder, given your research background, Jared, if we are talking about different types of eyetracking studies. For academic/generalizable research, I have found eyetracking studies to be pretty meaningless. Actually, that's pretty funny. I think the most exciting eye tracking stuff is happening in research. There were a ton of good posters and some neat presentations at CHI showing how eye tracking, as an alternative input device, could have some really cool applications, especially for accessibility. I also think there are some interesting cognitive and behavioral psych things to learn by using the devices. But I don't think there's been anything useful in terms of using it as a tool to enhance or inform the design process, so I'm guessing we agree there. > But for testing real products, and only trying to interpret results for those pages, it can be useful and not all that difficult, depending on the stimulus and tasks of course. Yah, not seeing that. What I see is that it falls nicely in the "If you can't dazzle 'em with your brilliance, feel free to baffle 'em with your bullshit" category of helping folks understand how to change their designs. But then again, what do I know? Jared Jared M. Spool User Interface Engineering 510 Turnpike St., Suite 102, North Andover, MA 01845 e: [EMAIL PROTECTED] p: 1 978 327 5561 http://uie.com Blog: http://uie.com/brainsparks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=28208 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Jared said: > "I said that I thought it is a voodoo technique. Deducing information > about a design from eyetracking is equivalent to reading tea leaves > and using a ouija board." > That's a pretty colorful exaggeration. Eyetracking lets you see where people are looking in real time. Without considering post-test analysis, this has real value in helping the facilitator better understand what is happening without interfering. One analogy I find useful, in terms of understanding what the participant is doing/thinking, is that having eyetracking versus not having eyetracking is like testing in person versus testing remotely. I wonder, given your research background, Jared, if we are talking about different types of eyetracking studies. For academic/generalizable research, I have found eyetracking studies to be pretty meaningless. But for testing real products, and only trying to interpret results for those pages, it can be useful and not all that difficult, depending on the stimulus and tasks of course. I also wonder if some people have been burned by past bad experiences with faulty eyetrackers and bad software. My lab at school had three separate eyetrackers and none of them worked correctly. The Tobii one that I use now is easy to use and the analysis software is very good. Paul On Sat, Apr 19, 2008 at 9:46 PM, Jared M. Spool <[EMAIL PROTECTED]> wrote: > I didn't say that I thought eye tracking was fluff. > > I said that I thought it is a voodoo technique. Deducing information > about a design from eyetracking is equivalent to reading tea leaves > and using a ouija board. > > The latter are cheaper, but just as reliable. > > Every person I know who swears by eyetracking and has stories on how > its helped them can't explain how they would've gotten the same > results if some other professional had looked at the same raw data. > Until we can get to that point, the reader of the data will be more > important than the data itself, thereby making tea leaf reading a > viable alternative. > > Jared > > Jared M. Spool > User Interface Engineering > 510 Turnpike St., Suite 102, North Andover, MA 01845 > e: [EMAIL PROTECTED] p: 1 978 327 5561 > http://uie.com Blog: http://uie.com/brainsparks > > > > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > Posted from the new ixda.org > http://www.ixda.org/discuss?post=28208 > > > > Welcome to the Interaction Design Association (IxDA)! > To post to this list ... [EMAIL PROTECTED] > Unsubscribe http://www.ixda.org/unsubscribe > List Guidelines http://www.ixda.org/guidelines > List Help .. http://www.ixda.org/help > Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
At the cog sci lab when I was in grad school the eye tracking equipment pretty much sat in the corner collecting dust. The general feeling was that we had leaned all we could from that technology years ago. We generally knowhow people see. It does have a great 'gee wiz' effect when you show and tell. All the execs think it is really cool. On Apr 20, 2008, at 9:06 AM, Andy Edmonds wrote: > Wow, what skepticism in this thread! I'll admit that I don't use > my eye > tracker as often as split testing, but I do feel compelled to offer a > more positive view on the matter. Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Wow, what skepticism in this thread! I'll admit that I don't use my eye tracker as often as split testing, but I do feel compelled to offer a more positive view on the matter. The #1 utility of the eye tracker, for me, is in helping me understand the user's cognition during a test session. With real time gaze data on a 2nd observer only screen, I don't have to work as hard on eliciting verbal protocol. I have also used the eye gaze reports to ask the user questions after the session -- a methodology others have developed more fully. Heck, I even spotted "button gravity" in my lab: http://flickr.com/photos/andyed/450579101/ Bruno: Regarding mouse movements, it's clear that eye movements are much higher signal, but mouse position has more data than twiddled fingers. I've summarized research on this on my blog and in a recent publication: http://alwaysbetesting.com/abtest/index.cfm/2007/4/29/Eye-Tracking-vs-Mouse-Tracking There's a longstanding and largely unsuccessful effort to generate quantitative quality metrics from eye-tracking data. That said, distance traveled by eye has been used productively in LukeW's work on forms, presented at Jared's Web App Summit recently. I've also been able to show good design leads to more efficient scan paths, http://flickr.com/photos/[EMAIL PROTECTED]/2175663626/. I won't dispute that many of the insights from eye-tracking are fairly obvious (ex. no headings in a long menu? http://flickr.com/photos/[EMAIL PROTECTED]/2177600531/), but there's something to be said for how well the visualizations engage consumers. To help with these basic types of insights, we've developed a vision simulation in a browser, "Stomper Scrutinizer" that helps reveal the multiple fixation requirements of left aligned form labels for example. Andy Bruno Figueiredo wrote: > Eye tracking is just like tracking mouse movement or clicks. It > doesn't really shows you what users are thinking, they're just > secondary manifestations of their thoughts. It's just like when you > twiddle your fingers on a table while thinking about what to do next. > It has nothing to do with it. Granted, there's some usefulness in the > data, since you can uncover some problems, but generally sitting with > a user and understanding it's train of thought is much more > insightful. > > > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > Posted from the new ixda.org > http://www.ixda.org/discuss?post=28208 > > > > Welcome to the Interaction Design Association (IxDA)! > To post to this list ... [EMAIL PROTECTED] > Unsubscribe http://www.ixda.org/unsubscribe > List Guidelines http://www.ixda.org/guidelines > List Help .. http://www.ixda.org/help > > Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
On Apr 19, 2008, at 2:46 PM, Jared M.Spool wrote: > Every person I know who swears by eyetracking and has stories on how > its helped them can't explain how they would've gotten the same > results if some other professional had looked at the same raw data. Nor can they explain why they wouldn't get *better* results and *better* recommendations from simply showing the UI to a half-decent user interface designer for 20 minutes. I've never seen a eyetracking recommendation that wasn't either (a) patently obvious to me ("Your 6-pixel-high light gray text should be made easier to see") or (b) completely stupid ("Move the search box to the left (where you currently have the picture of the cute little puppy) because everyone seems to spends time looking at the left side of the page"). -Cf Christopher Fahey Behavior http://www.behaviordesign.com 212.532.4002 x203 646.338.4002 mobile Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Speaking of failed pseudo-science - I had the unfortunate opportunity to see Ben Stein's Polemic Excretion "Expelled" on Friday eveningsigh On Sat, Apr 19, 2008 at 11:54 PM, Katie Albers <[EMAIL PROTECTED]> wrote: > At 10:27 PM -0400 4/19/08, Will Evans wrote: > >Yes, but Jared -- > >There are at least 2 full sessions @ June's UPA conference dedicated to > >eye-tracking, ergo it must be a valid technique! > > Not valid , but accepted. Surely we are all familiar with the > difference between those twoAnd I suspect you know it :) If > not...I know an excellent phrenologist > > Katie > > >Maybe next year we will > >have a "Tea-Leaf Reading Analytical Practices for Enhanced User > Experience," > >which will follow "Rapid A-B testing with Mescaline & Electroshock > Therapy: > >Getting a Charge Out of User Testing" session. > > > > > > > >On Sat, Apr 19, 2008 at 9:46 PM, Jared M. Spool <[EMAIL PROTECTED]> wrote: > > > >> I didn't say that I thought eye tracking was fluff. > >> > >> I said that I thought it is a voodoo technique. Deducing information > >> about a design from eyetracking is equivalent to reading tea leaves > >> and using a ouija board. > >> > >> The latter are cheaper, but just as reliable. > >> > >> Every person I know who swears by eyetracking and has stories on how > >> its helped them can't explain how they would've gotten the same > >> results if some other professional had looked at the same raw data. > >> Until we can get to that point, the reader of the data will be more > >> important than the data itself, thereby making tea leaf reading a > >> viable alternative. > >> > >> Jared > >> > >> Jared M. Spool > >> User Interface Engineering > >> 510 Turnpike St., Suite 102, North Andover, MA 01845 > >> e: [EMAIL PROTECTED] p: 1 978 327 5561 > >> http://uie.com Blog: http://uie.com/brainsparks > >> > >> > >> > >> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > >> Posted from the new ixda.org > >> http://www.ixda.org/discuss?post=28208 > >> > >> > >> > >> Welcome to the Interaction Design Association (IxDA)! > >> To post to this list ... [EMAIL PROTECTED] > >> Unsubscribe http://www.ixda.org/unsubscribe > >> List Guidelines http://www.ixda.org/guidelines > >> List Help .. http://www.ixda.org/help > >> > > > > > > > >-- > >~ will > > > >"Where you innovate, how you innovate, > >and what you innovate are design problems" > > > > >- > >Will Evans | User Experience Architect > >tel +1.617.281.1281 || [EMAIL PROTECTED] > > >- > > > >Welcome to the Interaction Design Association (IxDA)! > >To post to this list ... [EMAIL PROTECTED] > >Unsubscribe http://www.ixda.org/unsubscribe > >List Guidelines http://www.ixda.org/guidelines > >List Help .. http://www.ixda.org/help > > > -- > > > Katie Albers > [EMAIL PROTECTED] > > Welcome to the Interaction Design Association (IxDA)! > To post to this list ... [EMAIL PROTECTED] > Unsubscribe http://www.ixda.org/unsubscribe > List Guidelines http://www.ixda.org/guidelines > List Help .. http://www.ixda.org/help > -- ~ will "Where you innovate, how you innovate, and what you innovate are design problems" - Will Evans | User Experience Architect tel +1.617.281.1281 || [EMAIL PROTECTED] - Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
At 10:27 PM -0400 4/19/08, Will Evans wrote: >Yes, but Jared -- >There are at least 2 full sessions @ June's UPA conference dedicated to >eye-tracking, ergo it must be a valid technique! Not valid , but accepted. Surely we are all familiar with the difference between those twoAnd I suspect you know it :) If not...I know an excellent phrenologist Katie >Maybe next year we will >have a "Tea-Leaf Reading Analytical Practices for Enhanced User Experience," >which will follow "Rapid A-B testing with Mescaline & Electroshock Therapy: >Getting a Charge Out of User Testing" session. > > > >On Sat, Apr 19, 2008 at 9:46 PM, Jared M. Spool <[EMAIL PROTECTED]> wrote: > >> I didn't say that I thought eye tracking was fluff. >> >> I said that I thought it is a voodoo technique. Deducing information >> about a design from eyetracking is equivalent to reading tea leaves >> and using a ouija board. >> >> The latter are cheaper, but just as reliable. >> >> Every person I know who swears by eyetracking and has stories on how >> its helped them can't explain how they would've gotten the same >> results if some other professional had looked at the same raw data. >> Until we can get to that point, the reader of the data will be more >> important than the data itself, thereby making tea leaf reading a >> viable alternative. >> >> Jared >> >> Jared M. Spool >> User Interface Engineering >> 510 Turnpike St., Suite 102, North Andover, MA 01845 >> e: [EMAIL PROTECTED] p: 1 978 327 5561 >> http://uie.com Blog: http://uie.com/brainsparks >> >> >> >> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . >> Posted from the new ixda.org >> http://www.ixda.org/discuss?post=28208 >> >> >> >> Welcome to the Interaction Design Association (IxDA)! >> To post to this list ... [EMAIL PROTECTED] >> Unsubscribe http://www.ixda.org/unsubscribe >> List Guidelines http://www.ixda.org/guidelines >> List Help .. http://www.ixda.org/help >> > > > >-- >~ will > >"Where you innovate, how you innovate, >and what you innovate are design problems" > >- >Will Evans | User Experience Architect >tel +1.617.281.1281 || [EMAIL PROTECTED] >- > >Welcome to the Interaction Design Association (IxDA)! >To post to this list ... [EMAIL PROTECTED] >Unsubscribe http://www.ixda.org/unsubscribe >List Guidelines http://www.ixda.org/guidelines >List Help .. http://www.ixda.org/help -- Katie Albers [EMAIL PROTECTED] Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Yes, but Jared -- There are at least 2 full sessions @ June's UPA conference dedicated to eye-tracking, ergo it must be a valid technique! Maybe next year we will have a "Tea-Leaf Reading Analytical Practices for Enhanced User Experience," which will follow "Rapid A-B testing with Mescaline & Electroshock Therapy: Getting a Charge Out of User Testing" session. On Sat, Apr 19, 2008 at 9:46 PM, Jared M. Spool <[EMAIL PROTECTED]> wrote: > I didn't say that I thought eye tracking was fluff. > > I said that I thought it is a voodoo technique. Deducing information > about a design from eyetracking is equivalent to reading tea leaves > and using a ouija board. > > The latter are cheaper, but just as reliable. > > Every person I know who swears by eyetracking and has stories on how > its helped them can't explain how they would've gotten the same > results if some other professional had looked at the same raw data. > Until we can get to that point, the reader of the data will be more > important than the data itself, thereby making tea leaf reading a > viable alternative. > > Jared > > Jared M. Spool > User Interface Engineering > 510 Turnpike St., Suite 102, North Andover, MA 01845 > e: [EMAIL PROTECTED] p: 1 978 327 5561 > http://uie.com Blog: http://uie.com/brainsparks > > > > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > Posted from the new ixda.org > http://www.ixda.org/discuss?post=28208 > > > > Welcome to the Interaction Design Association (IxDA)! > To post to this list ... [EMAIL PROTECTED] > Unsubscribe http://www.ixda.org/unsubscribe > List Guidelines http://www.ixda.org/guidelines > List Help .. http://www.ixda.org/help > -- ~ will "Where you innovate, how you innovate, and what you innovate are design problems" - Will Evans | User Experience Architect tel +1.617.281.1281 || [EMAIL PROTECTED] - Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Have seen the phenomenon where because someone is using "Eye Tracking Equipment" they are automatically given more status or overused because of the "coolness factor" of just having the equipment (independent of what the results mean and how other methods can help compliment the results) Also good thread about it here - http://www.ixda.org/discuss.php?post=22825&search=eye+tracking rgds, -- Daniel Szuc Principal Usability Consultant Apogee Usability Asia Ltd www.apogeehk.com Usability in Asia The Usability Kit - www.theusabilitykit.com On 19 Apr 2008, at 6:46 PM, Jared M.Spool wrote: > I didn't say that I thought eye tracking was fluff. > > I said that I thought it is a voodoo technique. Deducing information > about a design from eyetracking is equivalent to reading tea leaves > and using a ouija board. > > The latter are cheaper, but just as reliable. > > Every person I know who swears by eyetracking and has stories on how > its helped them can't explain how they would've gotten the same > results if some other professional had looked at the same raw data. > Until we can get to that point, the reader of the data will be more > important than the data itself, thereby making tea leaf reading a > viable alternative. > > Jared > > Jared M. Spool > User Interface Engineering > 510 Turnpike St., Suite 102, North Andover, MA 01845 > e: [EMAIL PROTECTED] p: 1 978 327 5561 > http://uie.com Blog: http://uie.com/brainsparks > > > > . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . > Posted from the new ixda.org > http://www.ixda.org/discuss?post=28208 > > > > Welcome to the Interaction Design Association (IxDA)! > To post to this list ... [EMAIL PROTECTED] > Unsubscribe http://www.ixda.org/unsubscribe > List Guidelines http://www.ixda.org/guidelines > List Help .. http://www.ixda.org/help Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
I didn't say that I thought eye tracking was fluff. I said that I thought it is a voodoo technique. Deducing information about a design from eyetracking is equivalent to reading tea leaves and using a ouija board. The latter are cheaper, but just as reliable. Every person I know who swears by eyetracking and has stories on how its helped them can't explain how they would've gotten the same results if some other professional had looked at the same raw data. Until we can get to that point, the reader of the data will be more important than the data itself, thereby making tea leaf reading a viable alternative. Jared Jared M. Spool User Interface Engineering 510 Turnpike St., Suite 102, North Andover, MA 01845 e: [EMAIL PROTECTED] p: 1 978 327 5561 http://uie.com Blog: http://uie.com/brainsparks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=28208 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Eye tracking is just like tracking mouse movement or clicks. It doesn't really shows you what users are thinking, they're just secondary manifestations of their thoughts. It's just like when you twiddle your fingers on a table while thinking about what to do next. It has nothing to do with it. Granted, there's some usefulness in the data, since you can uncover some problems, but generally sitting with a user and understanding it's train of thought is much more insightful. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=28208 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Eye tracking is very expensive for what it delivers, and, as with all analytics, data interpretation can be difficult because you know nothing about user motivation or intent. IMHO, analytics tools like session recording and click maps can provide somewhat similar data for a MUCH lower cost, both in terms of price and effort. -eva -- http://www.linkedin.com/in/kaniasty Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Thanks for sending this Andre, really interesting post. I was watching a user session this week in one room at the Ux agency we use and the moderator announced that, in the second room, they had setup eye tracking so we could observe that. Everyone else rushed through to watch it and I sat where I was as I have never gained anything useful from eye-tracking in real-time. That said, working out why someone has missed a header or navigation element due to an imbalance in perceived-affordance or other visual priority is made somewhat easier by the aggregated results of eye-tracking. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andre Charland Sent: 18 April 2008 00:11 To: [EMAIL PROTECTED] Subject: [IxDA Discuss] Is Eye Tracking too expensive or complicated? Hey All, I just put together about Eye Tracking (http://www.insideria.com/2008/04/is-eye-tracking-out-of-reach.html) but I thought I'd put the question out to the list: "Why isn't eye tracking used more in design and testing of rich internet applications?" Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
Eye-tracking has the same trap as all data collection - if you're not sure how the data will enable you to make better decisions, having reems of it doesn't help much. Depending on the kind of UI being designed, different levels of data are more useful than others. Eye-tracking gives you information about *everything* the user looks at in mili-second increments - will that be more useful in recognizing issues and evaluating a design than standard measures like time on task, errors, and success rates for tasks? Often the answer is no. There are plenty of other usability data collection methods thave have specialized use: for example keystroke level recording and GOMS analysis also provide highly detailed data that is great for some design problems but overkill for most. I don't know of a single design firm or software company that has invested in eye-tracking that uses the tool regularly. In special cases it's poweful (Say a Heads-up display for the F-17, or Mac Finder, complex high use items where miliseconds matter), but generally it provides data that doesn't much help answer the design and research questions designers have. There are some compilations of general findings from eye-tracking, but as interesting as they are they don't suggest changes or improvements so much as offer validation of concepts many designers already use. http://www.poynterextra.org/eyetrack2004/main.htm. It's great that someone has done this, and it's cool looking data, but the information found tends to be general enough that few are motivated to repeat the studies to see if their specific web-pages or software deviates from these typical saccade patterns. Lastly, eyetracking tools are flat out expensive. This means it's harder to find people experienced with the tools, and it takes more time and money to interpret data that comes out of those studies. -Scott Scott Berkun www.scottberkun.com - Original Message - From: "Andre Charland" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, April 17, 2008 3:11 PM Subject: [IxDA Discuss] Is Eye Tracking too expensive or complicated? Hey All, I just put together about Eye Tracking (http://www.insideria.com/2008/04/is-eye-tracking-out-of-reach.html) but I thought I'd put the question out to the list: "Why isn't eye tracking used more in design and testing of rich internet applications?" Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
An interesting question, which begs for me to plug something I am co-presenting at UPA entitled, "Practical Eyetracking." https://www.usabilityprofessionals.org/conference/2008/program/friday.htm If Jared Spool has been paraphrased correctly, I disagree that eyetracking is just fluff. As Rob Tannen hinted at, eyetracking can be very helpful in determining why things happened. Usually we have intuition do that, but often it is not obvious. Here's one example, we tested a design where people couldn't find the link to login to the site, even though it was in the main navigation. Eyetracking showed very clearly that people scanned the first few items in the navigation and then moved on without seeing the link. Seeing exactly where they stopped looking helped us understand why. And having a visual to show it helped us convince the client to change their design. Without eyetracking, we might have thought that the label was simply wrong. The ROI is a separate discussion, but there are two things holding eyetracking back: cost and the additional time it takes to plan studies and to analyze the findings. Compared to alternatives, the ROI is pretty low. But if you have one already, or have a bigger budget, it can be very useful. And don't discount impressing the execs with great visuals. Paul Nuschke Electronic Ink On Thu, Apr 17, 2008 at 7:11 PM, Andre Charland <[EMAIL PROTECTED]> wrote: > Hey All, > > I just put together about Eye Tracking > (http://www.insideria.com/2008/04/is-eye-tracking-out-of-reach.html) > but I thought I'd put the question out to the list: > > "Why isn't eye tracking used more in design and testing of rich > internet applications?" > > I really have no experience with it, but am going to be working with > it this summer so looking for thoughts from some of you more > experienced with usability testing and eye tracking. I think for me > it's been cost and lack of knowledge/acccess that's kept me away from > it in the past. > > Cheers! > > -- > André Charland > President and Co-Founder, Nitobi > e. [EMAIL PROTECTED] > skype. ebadre > b. http://blogs.nitobi.com/andre > w. http://www.nitobi.com > > Welcome to the Interaction Design Association (IxDA)! > To post to this list ... [EMAIL PROTECTED] > Unsubscribe http://www.ixda.org/unsubscribe > List Guidelines http://www.ixda.org/guidelines > List Help .. http://www.ixda.org/help > Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
...oops I meant Design B would be the better option (assuming less visual workload is preferable). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=28208 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
It does have value as a secondary diagnostic tool. In the context of usability testing, eye tracking does not determine the presence of a usability problem, but helps determine what led to that problem in conjunction with performance data, faciliator observations and user self-reporting. For example, different people may fail a task for different reasons that eye tracking can reveal - overlooking a critical instruction versus reading it but failing to understand it. In some cases users can tell you this reliably, in others they can't. Also, eye tracking provides a comparative metric between designs that are equivalent on other performance measures. For example, Design A may require greater visual scanning or workload than Design B, so all other things being equal, Design A might be the better option. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=28208 Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
Re: [IxDA Discuss] Is Eye Tracking too expensive or complicated?
At the Information Architecture conference in Miami last week, Jared Spool imparted that eye tracking is just fluff to get the executives excited about UCD he doesn't value it as a science beyond the obvious. We already know that people don't look at advertisements and avoid certain areas of the page. While it will look good on a resume to have the experience, I tend to agree with Jared. (Embedded image moved to file: pic05097.gif) "Andre Charland" <[EMAIL PROTECTED] mail.com> To Sent by: [EMAIL PROTECTED] [EMAIL PROTECTED] cc ists.interactiond esigners.com Subject [IxDA Discuss] Is Eye Tracking too expensive or complicated? 04/17/2008 07:11 PM Hey All, I just put together about Eye Tracking (http://www.insideria.com/2008/04/is-eye-tracking-out-of-reach.html) but I thought I'd put the question out to the list: "Why isn't eye tracking used more in design and testing of rich internet applications?" I really have no experience with it, but am going to be working with it this summer so looking for thoughts from some of you more experienced with usability testing and eye tracking. I think for me it's been cost and lack of knowledge/acccess that's kept me away from it in the past. Cheers! -- André Charland President and Co-Founder, Nitobi e. [EMAIL PROTECTED] skype. ebadre b. http://blogs.nitobi.com/andre w. http://www.nitobi.com Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help - CONFIDENTIALITY STATEMENT: This e-mail transmission contains information that is intended to be confidential. It is intended only for the addressee named above. If you receive this e-mail in error, please do not read, copy, or disseminate it. If you are not the intended recipient, any disclosure, copying, distribution or use of the contents of this information is prohibited. Please reply to the message immediately by informing the sender that the message was misdirected. After replying, please erase it from your computer system. Your assistance in correcting this error is appreciated.<> Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help
[IxDA Discuss] Is Eye Tracking too expensive or complicated?
Hey All, I just put together about Eye Tracking (http://www.insideria.com/2008/04/is-eye-tracking-out-of-reach.html) but I thought I'd put the question out to the list: "Why isn't eye tracking used more in design and testing of rich internet applications?" I really have no experience with it, but am going to be working with it this summer so looking for thoughts from some of you more experienced with usability testing and eye tracking. I think for me it's been cost and lack of knowledge/acccess that's kept me away from it in the past. Cheers! -- André Charland President and Co-Founder, Nitobi e. [EMAIL PROTECTED] skype. ebadre b. http://blogs.nitobi.com/andre w. http://www.nitobi.com Welcome to the Interaction Design Association (IxDA)! To post to this list ... [EMAIL PROTECTED] Unsubscribe http://www.ixda.org/unsubscribe List Guidelines http://www.ixda.org/guidelines List Help .. http://www.ixda.org/help