Re: Multi-Touch
With that being said I have been thinking about multi-touch lately and after some things on the wiki page http://wiki.openmoko.org/wiki/UI_Improvements I noticed that is mentioned the zoom feature of the iphone. as an example. The aim of the question is to discover how a monotouch screen behaves when you try to use it as multi touch: is it determinist, or erratical? If somebody could try reproducing various strange usages of the screen and report any deterministic/detectable behaviour, we might find ourselves additional input methods based on guess hacks. For instance, i think we can use cursor jumps as inputs; example: P**+ * 2 * ** * 1* - **N Zoom + P **+ * 1 * * * *2 * - **N Next page/item It would be some timing-based detection, like non-linear handgesture recognition, done with 1 or 2 fingers. We have to: 1- experiment the touchscreen with all imaginable gestures (ex: 2 finger sliding, ...) 2- select the interesting behaviours 3- evaluate their determinism, eventually low level hacks on the kernel module 4- integrate them into openmoko finger-based controls So my question now is do we have access to the bounding box? If we can get at the coordinates of the bounding box can we not figure out if the bounding box is shrinking or growing? Yes, that is an open question :) Also if we have access to the bounding box we could with some practice figure out when users are trying to rotate an image. Indeed: if the bounding box is accurate, what if: - you narrow the 2 fingers? - zoom - rotate the box? rotation / linear map rotation - 2 parallel fingers (horizontal) sliding down = flat bounding box (almost a line) = scrolling down ... But, i'm not sure i understood what a bouncing box is: if all of this was true, then the touchscreen would report 2 points / an area... It would be great if people having access to hardware could try doing some detailed testing / reporting. But, it can wait until the next shipping phase is over :) the high resolution of the neo this should help a great deal. Well, even with high resolution, our eyes don't have better zooming :) Especially on mobility situation... . You press the zoom button then click an area to enlarge. Just curious how difficult would this be? Well, it's quite easily doable if the webbrowser / image displayer implements zooming ! looking at the iphone and it doesn't seem to be very multi touch but as it has not come out I could be wrong. It seems to only be able to accept 2 simultaneous inputs. Hey, that's just what's needed :) How many free space do you have on such a screen? Ciao Florent ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: Multi-Touch
On 4/3/07, adrian cockcroft [EMAIL PROTECTED] wrote: This is key:- Pressure has almost no effect on a single touch, but not so on a double touch. The relative pressures will cause a significant skewing effect towards the harder touch. You can easily move the pointer along the line between your two fingers by changing the relative pressure. So we will not see clearly defined bounding box limits. The point will skate around within the limits depending on relative pressure. So I guess we need someone with a device to test this and see how much pressure actually affects the neo touch screen. With that information I think we could see how easy it would be to get an average bounding box approximation.. The first finger will set a clear start point, the second finger will make that point shoot off towards it, but it will not go all the way to the second touch. The effect should be to oscillate along the line between the two end points, and it wont return to the position of the first touch. If we capture a clear single touch, and an average position of oscillation, then we can take the average oscillation to be the center of the bounding box, and project an estimate of the opposite corner where the second touch should be. With the right filtering and limiting algorithm it should be possible to get the effect we want. If we can give visual feedback on the screen showing the touch points and bounding box it may help the user control the input better. Ok so what if this feature was disabled by default. Since enabling it might slow some functionality. When the user enables the feature he/she will have to go through a config which does a calibration. The user runs through several scenarios where the program can gather the relative pressure difference in known circumstance with the desired result known as well. It could then store that information in a database based on which user is using it, if there are multi users, if not just store it in a config file. Challenges: In comparison with a true dual touch input device, its going to react more slowly as the algorithm will need to gather more data to decide where the pointer should be. Some of the faster moving single touch gestures may be hard to distinguish from multi touch. Adrian I agree, but it would still be a nice feature to have and I could deal with a little lag for added feature especially if I could enable it or disable it. With the single touch fast gestures we could set that up inside of the calibration also. That way if it thinks it's multi touch it compares it with it's database which links it to the single touch fast gesture instead. It would be slow but you could disable the multi touch option and then single gestures would be fast again. Or we could have the multi touch be enabled in certain programs like web browsing or picture viewing, and disabled for the rest. But I think a simple button in the header or footer should work alright. You could have it turn green when active and red when deactivated. ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: Multi-Touch
Two more thoughts... If you hold down one finger and tap the other one, the mouse pops over and back again. This could be a way to set a bounding box or turn on the mode. We may be able to set the sensitivity and sampling rate much higher than normal to support a more advanced algorithm. To test this, we could capture and share some raw (time,x,y) event streams from a touch screen, and try processing them offline. Then we don't need actual hardware (a spreadsheet is may even be good enough) to figure out algorithms. Adrian On 4/3/07, mathew davis [EMAIL PROTECTED] wrote: On 4/3/07, adrian cockcroft [EMAIL PROTECTED] wrote: This is key:- Pressure has almost no effect on a single touch, but not so on a double touch. The relative pressures will cause a significant skewing effect towards the harder touch. You can easily move the pointer along the line between your two fingers by changing the relative pressure. So we will not see clearly defined bounding box limits. The point will skate around within the limits depending on relative pressure. So I guess we need someone with a device to test this and see how much pressure actually affects the neo touch screen. With that information I think we could see how easy it would be to get an average bounding box approximation.. The first finger will set a clear start point, the second finger will make that point shoot off towards it, but it will not go all the way to the second touch. The effect should be to oscillate along the line between the two end points, and it wont return to the position of the first touch. If we capture a clear single touch, and an average position of oscillation, then we can take the average oscillation to be the center of the bounding box, and project an estimate of the opposite corner where the second touch should be. With the right filtering and limiting algorithm it should be possible to get the effect we want. If we can give visual feedback on the screen showing the touch points and bounding box it may help the user control the input better. Ok so what if this feature was disabled by default. Since enabling it might slow some functionality. When the user enables the feature he/she will have to go through a config which does a calibration. The user runs through several scenarios where the program can gather the relative pressure difference in known circumstance with the desired result known as well. It could then store that information in a database based on which user is using it, if there are multi users, if not just store it in a config file. Challenges: In comparison with a true dual touch input device, its going to react more slowly as the algorithm will need to gather more data to decide where the pointer should be. Some of the faster moving single touch gestures may be hard to distinguish from multi touch. Adrian I agree, but it would still be a nice feature to have and I could deal with a little lag for added feature especially if I could enable it or disable it. With the single touch fast gestures we could set that up inside of the calibration also. That way if it thinks it's multi touch it compares it with it's database which links it to the single touch fast gesture instead. It would be slow but you could disable the multi touch option and then single gestures would be fast again. Or we could have the multi touch be enabled in certain programs like web browsing or picture viewing, and disabled for the rest. But I think a simple button in the header or footer should work alright. You could have it turn green when active and red when deactivated. ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: Multi-Touch
Yy ! We may have found something that's 100% compatible with the fingerwheel. Let's do it this way: - this socalled first touch can be done on the mokowheel itself: put your left thumb on the black area; there you enter fake multitouchscreen mode - keeping this finger in the black circle, touch another place of the screen : the cursor moves (not instantly if what we guessed is true) at a *constant* speed; or it warps, if it's implementation-dependant (is suspect the low level driver to report point A - point B as a jump) - if we can calculate the speed of the cursor every moment, we can identify a down left+second finger; either way we can detect warping - afterwards, * slide your righthand finger down, it scrolls up * slide your righthand finger up, it scrolls down * slide it left, next page/item * slide it right, previous page/item * do a circle: rotation * narrow towards the black circle: zoom - * go away: zoom + - when you leave the black circle, it's out of finger operational mode What's really great about it, is when you're looking at a map, you can rotate and zoom at the same time. The ownly downside is that we MUST use both hands (or put the device on a table) I'm gonna do mockups... Florent ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: built-in scripting languages.
On 4/3/07, Bryan Larsen [EMAIL PROTECTED] wrote: [...] However, developers who choose one of these languages for their applications will not be able to see their applications included in the standard ROM nor available for use by those without an external microSD card. [...] I'm not sure why you need an external MicroSD card, but all Neo's come with a 512 MB MicroSD card. Sean mentioned that in the FOSDEM presentation. Also, see http://wiki.openmoko.org/wiki/FAQ#Q:_How_much.3F -Steven ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: Multi-Touch
Mockups done (ugly, but gives an idea) http://wiki.openmoko.org/wiki/UI_Improvements#Handgesture_recognition_proposal In fact, we may not even need the fake multi touch thing, if we define a finger-mode zone. Just put your first finger on the wheel, and the second one has handgesture enabled... ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: built-in scripting languages.
Steven ** wrote: I'm not sure why you need an external MicroSD card, but all Neo's come with a 512 MB MicroSD card. Sean mentioned that in the FOSDEM presentation. Also, see http://wiki.openmoko.org/wiki/FAQ#Q:_How_much.3F Sure, the developer's kit comes with a microSD card. I expect all developer's to have one. I'm worried about end users. I don't expect all user phones to come with a micro SD card. I also don't expect phones to come with firmware installed on the micro SD cards. I expect users to balk when installing small applications that take up precious megabytes of space because a large scripting language is a prerequisite. I expect users to want to reserve their microSD cards for large music and video files. Bryan ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: Multi-Touch
Last post, promise :) I found this: http://iscroll2.sourceforge.net/ iScroll2 is a modified trackpad driver that adds two-finger scrolling capabilities to supported pre-2005 PowerBooks and iBooks on OS X 10.3 and up. To scroll, just place two fingers on your trackpad instead of one. Both fingers need to be placed next to each other horizontally (*not* vertically, the trackpad cannot detect that). Some people get better results with their finger spaced a little bit apart, while others prefer having the fingers right next to each other. iScroll2 provides two *scrolling modes*: Linear and circular scrolling. For *linear scrolling*, move the two fingers up/down or left/right in a straight line, respectively, to scroll in that direction. *Circular scrolling* works in a way similar to the iPod's scroll wheel: Move the two fingers in a circle to scroll up or down, depending on whether you move in a clockwise or counterclockwise direction. Added here: http://wiki.openmoko.org/wiki/UI_Improvements#Extending_the_touchscreen_capabilities_and_input_methods Cheers And good night Florent ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: Multi-Touch
I personally like the configuration of my synaptics touch pad. I have left, middle, and right clicks (1, 2, and 3 finger tapping, respectively), and I can also use the right and bottom edges for vertical and horizontal scrolling. However, keep in mind that touch *screens* are very different in design than touch *pads*. With a touch screen, the cursor moves to the coordinates of the pressure. With a touch pad, the cursor does not move based solely on pressure being applied. Instead, touch pads track the motion of the pressure point and move the cursor in the appropriate direction and speed. Mixing and matching the capabilities of these may be more difficult than we think, not just from a hardware/driver point of view, but also in user-friendliness. ~Bradley Florent THIERY wrote: Last post, promise :) I found this: http://iscroll2.sourceforge.net/ iScroll2 is a modified trackpad driver that adds two-finger scrolling capabilities to supported pre-2005 PowerBooks and iBooks on OS X 10.3 and up. To scroll, just place two fingers on your trackpad instead of one. Both fingers need to be placed next to each other horizontally (*not* vertically, the trackpad cannot detect that). Some people get better results with their finger spaced a little bit apart, while others prefer having the fingers right next to each other. iScroll2 provides two *scrolling modes*: Linear and circular scrolling. For *linear scrolling*, move the two fingers up/down or left/right in a straight line, respectively, to scroll in that direction. *Circular scrolling* works in a way similar to the iPod's scroll wheel: Move the two fingers in a circle to scroll up or down, depending on whether you move in a clockwise or counterclockwise direction. Added here: http://wiki.openmoko.org/wiki/UI_Improvements#Extending_the_touchscreen_capabilities_and_input_methods Cheers And good night Florent ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: built-in scripting languages.
Steven ** wrote: On 4/3/07, Bryan Larsen [EMAIL PROTECTED] wrote: [...] However, developers who choose one of these languages for their applications will not be able to see their applications included in the standard ROM nor available for use by those without an external microSD card. [...] I'm not sure why you need an external MicroSD card, but all Neo's come with a 512 MB MicroSD card. Sean mentioned that in the FOSDEM presentation. Also, see It would be very nice if 'core' applications fitted on the internal NAND flash. If they don't, then this essentially means that users can't easily change out their SD cards. Being utterly unable to power down your phone, and swap the cards over from your camera as you've forgotten the lead, to send a picture to someone, for example would be really annoying. ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: Multi-Touch
mathew davis wrote: snip * The output is the center of the bounding box of the touched area * The touch point skips instantly on double touch * Pressure has almost no effect on a single touch, but not so on a double touch. The relative pressures will cause a significant skewing effect towards the harder touch. You can easily move the pointer along the line between your two fingers by changing the relative pressure. So my question now is do we have access to the bounding box? If we can get at the coordinates of the bounding box can we not figure out if the bounding box is shrinking or growing? The iphone has a camera which Ok - I wrote the above, and it could do with some clarification. The 'center of the bounding box' is the only thing the current hardware has access to. It is 'computed' solely in the resistances of the touchscreen, not externally. Pressing harder with one finger than the other, or pressing over a larger area gives a (possibly slightly, possibly greatly, depending on design) skew towards the area with the higher contact pressure * area. (And it may not be as simple as contact pressure*area, it probably hits a limit at a small pressure) When measuring the touch position this is basically what is done. The screen is composed of two resistive layers, one flexible plastic, one glass, with linear electrodes on the X or Y axes (parallel electrodes on the same layer). Connect X+ and X- to battery + and -. This now creates a voltage gradient vertically across the screen - say 0V at the bottom, and 1V at the top. Now, when someone presses the screen at a point, this is connected to the 'Y' electrodes, and the voltage (say 0.5V) can be read out and converted into 'X=320'. (repeat for Y+ Y- and an X for other axes) If the screen is pressed over multiple points, or areas, it's a little more complex. You have now current flowing in addition to through the X layer, flowing parallel to the X layer, through the Y layer - this will distort the nice smooth gradient, and make it so that instead of rising smoothly from 0 to 1V as you go up the display, it rises half as steeply over the touched area. This assumes that the contact resistance is zero though, and it's not. As the resistance at a touch point drops, the sensed point moves towards that point, A not-terrible mechanical analogy is to take a hacksaw blade, and put it on a towel. pressing (gently) on it on it at several points will (usually) make only one point furthest down. ___ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
