from:"Kornel"

I think it'd be unfortunate if this API had just one shared lock per browsing 
context and required components on the page to coordinate locking, but provided 
no means to do so. This will force authors of libraries and components to 
create dummy iframes just to have their private lock, and libraries/pages 
without such workaround will be messing up each other's locks.

Having just a single shared DOM0-style event handler navigator.wakeLock.onlost 
looks especially jarring. I would expect this to be a proper DOM event that can 
be used with normal addEventListener (please avoid repeating the mistake of 
matchMedia).


To make some coordination possible, the simplest method could be to keep track 
of number of lock requests and releases, like retain/release in Objective-C:

navigator.wakeLock.request(screen); // locks
navigator.wakeLock.request(screen); // increases lock count
navigator.wakeLock.release(screen); // not released yet, but decreases lock 
count
navigator.wakeLock.release(screen); // now released for real

However, as you probably know from Objective-C, perfect balancing of 
retain/release takes care and discipline. Personally, I wouldn't trust all 3rd 
party libraries/widgets/ads to be careful with this. In fact, I expect some 
clever libraries to ruin this with:

while(navigator.wakeLock.isHeld(screen)) 
navigator.wakeLock.release(screen); // just release the damn thing in my 
leaky code!


Therefore, if WakeLock needs to be purely JS API, I strongly prefer having 
WakeLock available only as an object instance, but without exposing GC 
behavior—if it's lost, it's like a missing release call. 

If devtools ever get monitoring of unhanded errors in Promise objects, they 
could also warn against lost WakeLock objects—it's the same type of problem 
dependent on GC.

I'm assuming that release would work only once on each lock object:

var lock = new WakeLock(screen);
lock.release();
lock.release(); // ignored, so it doesn't unlock any other component's lock

This makes coordination easier: each page component can easily create their own 
lock independently (without needing to create an iframe to get their own lock), 
and can't release any other component's lock.


-- 
regards, Kornel

[whatwg] Feature-detectable WakeLocks

WakeLock.request() expecting a string isn't very friendly to feature detection. 

I'd prefer if individual lock types were instances of objects, e.g. 
navigator.*Lock objects could be instances of a variant of the WakeLock 
interface:

navigator.screenLock.request();
navigator.screenLock.isHeld();

navigator.cpuLock.request();
navigator.cpuLock.release();


Alternatively, if the WakeLock was instantiable (to have a standard way for 
independent page components to share locks) then these objects could be 
constructors:

if (navigator.ScreenLock) {
var lock = new navigator.ScreenLock();
…
lock.release();
}

(or `new navigator.wakeLocks.Screen()`, etc.)

Having specific instances for different types of locks could also enable 
elegant extensibility of the API, e.g.

var screenLock = new navigator.ScreenLock();
screenLock.dimScreen(); // completely made-up API

var cpuLock = new navigator.CpuLock();
cpuLock.setThreadPriority(low); // completely made-up API

-- 
regards, Kornel

[whatwg] Preventing wake lock leaks with DOM nodes

My biggest concern with the WakeLock API is that it's easy to forget (or fail) 
to release the lock.

It's not a problem with the API per se, but a programming problem in general: 
resource management in non-trivial programs is hard. WakeLocks are especially 
problematic in this regard, as a leaked lock won't cause any immediate 
problems for the developer, so this type of bug can easily go unnoticed.

So I think lifetime of WakeLocks needs to be attached to something visible to 
make failure to release the lock immediately obvious.


In case of screen lock it's especially easy: the whole purpose of this lock is 
to keep something visible on screen, so we can require that something to be 
explicitly connected to the lock.

For example if I were creating a widget that displays a presentation on the 
page, I could attach the screen lock to the canvas or svg element that 
holds the presentation:

new navigator.ScreenLock(myCanvas);

and if the canvas was removed from the document or hidden in any way, then the 
browser could turn the screen off as usual, and I wouldn't have to do anything!

It's nearly impossible to forget to remove a visible DOM element from the 
document — the mistake is likely to be quite obviously visible. If screen lock 
lifetime was dependent on visibility of a DOM element, then it would also be 
very hard to leak the lock without noticing it!

(that's a variant of wake-lock:display CSS proposal, but less explicitly 
dependent on CSS).


With CPU lock it's less clear cut. I think tying it to a notification may be a 
good idea. Alternatively, perhaps the lock itself could be an instance of the 
progress element that author is supposed to insert to the document? ;)

-- 
regards, Kornel

Re: [whatwg] Proposal: Wake Lock API

 On Monday, August 18, 2014 at 6:24 PM, Kornel Lesiński wrote:
 
 I think it'd be unfortunate if this API had just one shared lock per 
 browsing context and required components on the page to coordinate locking, 
 but provided no means to do so.  
 
 
 The API allows scripts to check which locks are currently held (as either a 
 `isHeld()` or `getCurrentLocks()`, for which I just sent a PR for).   

I don't understand how is that helping.

Let's say I have embedded a Slideshare presentation and a YouTube video on my 
page. I start watching slides, then start playing the video, then finish 
watching slides.

When Slideshare finishes and wants to release the lock, it can't learn via this 
API whether YouTube still wants the lock. When Slideshare started isHeld was 
false, but setting it back to that original state would be incorrect. When 
Slideshare finished isHeld was true, but that doesn't tell anything either, 
since Slideshare itself set it to true.


The only way I see for coordinating lock between independent components on the 
page is not via isHeld(), but by defensively re-setting the lock.

In my previous example both Slideshare and YouTube would have to watch for 
'lost' events (but not via the Netscape-style onlost footgun!) and keep 
re-requesting the lock soon after it's been released, for as long as they need 
it.

IMHO that's really ugly.

If re-requesting is supposed to be the pattern for maintaining locks properly, 
then the whole API could be cut down to just events:

window.addEventListener('beforeScreenLock', function(e) {
if (stillShowingStuff) e.preventDefault();
}, false);

The browser would fire beforeScreenLock event every time the OS is about to 
turn the screen off. To keep the screen on for another while the page just 
needs to prevent the event.


 This will force authors of libraries and components to create dummy iframes 
 just to have their private lock, and libraries/pages without such workaround 
 will be messing up each other's locks.
 
 Currently, iframes are not allowed to have locks - only top-level browsing 
 contexts are. This is to avoid things like embedded ads from requesting wake 
 locks.   

That's a noble goal. However, it may not be effective against ads in practice, 
because majority of ads are embedded using script. OTOH it may prevent use of 
locks in well-behaved embedded documents (slideshows and the like). 

At very least I suggest controlling this like other potentially-annoying 
features via iframe sandbox=allow-locks.

 To make some coordination possible, the simplest method could be to keep 
 track of number of lock requests and releases, like retain/release in 
 Objective-C:
 
 navigator.wakeLock.request(screen); // locks
 navigator.wakeLock.request(screen); // increases lock count
 navigator.wakeLock.release(screen); // not released yet, but decreases 
 lock count
 navigator.wakeLock.release(screen); // now released for real
 
 
 In my dummy implementation, I've just been using (the Swift equivalent of):
 [UIApplication sharedApplication].idleTimerDisabled = YES;

I admire elegance of most of Cocoa's APIs, but I don't think design of that one 
is worth copying. It's global shared mutable state, and there's even negation 
in the property name—it may have been a quick hack that slipped though Cocoa QA 
(the documentation even includes a warning that it should be used with care and 
only when necessary).

Apple has tight control over what apps do, and iOS apps generally don't have as 
many code mashups as web pages, so such primitive fragile API may be less of a 
concern for Apple.


Other APIs you've liked to 
(https://developer.apple.com/library/mac/qa/qa1340/_index.html) are better:

IOPMAssertionCreateWithName/IOPMAssertionRelease creates instances that can be 
individually independently released, so each component can have its own lock 
and they can't release each other's locks accidentally (similar to the `new 
WakeLock()` proposal).

IORegisterForSystemPower uses a callback (similar to the 
`event.preventDefault()` solution I've outlined earlier in this mail).

 This makes coordination easier: each page component can easily create their 
 own lock independently (without needing to create an iframe to get their own 
 lock), and can't release any other component's lock.
 Personally, I don't know if I agree that it makes coordination easier. Seems 
 that having a centralized place to check what is currently being held makes 
 life a lot easier, because it allows scripts to check if they actually need 
 to request a lock or not.

No! That's a footgun! You can't do `if (!isHeld()) request()`, because the 
other component holding the lock may have shorter lifetime than yours, and you 
can't know that.

-- 
regards, Kornel

Re: [whatwg] Feature-detectable WakeLocks


 I'd prefer if individual lock types were instances of objects, e.g. 
 navigator.*Lock objects could be instances of a variant of the WakeLock 
 interface:
 
 navigator.screenLock.request();
 navigator.screenLock.isHeld();
 
 navigator.cpuLock.request();
 navigator.cpuLock.release();
 
 Personally, this doesn't strike me as good API design. It means having a 
 bunch of attributes that all use the same class but only differ in name.

Really?

I think clearly separating different classes of locks (with a common base 
class) is much better than conflating them behind a weakly typed string-driven 
API.

It's like:
Element.firstChild.getAttribute(…);
Element.nextSibling.getAttribute(…);
 
instead of:
Element.getAttribute(firstChild, …);
Element.getAttribute(nextSibling, …);

 Alternatively, if the WakeLock was instantiable (to have a standard way for 
 independent page components to share locks) then these objects could be 
 constructors:
 
 if (navigator.ScreenLock) {
 var lock = new navigator.ScreenLock();
 …
 lock.release();
 }
 
 (or `new navigator.wakeLocks.Screen()`, etc.)
 We don't have any APIs like this today on the Web.  It would be weird :)  

Weird is subjective and a vague criticism. Can you elaborate what's wrong 
with that?

 It would just be better to have a constructor on the interface: `new 
 WakeLock(screen)` or whatever.  

I don't see any benefit in obscuring types of the objects. 

String-driven API doesn't allow simple feature detection, and a single type 
that conflates all lock types makes extensibility uglier. You won't be able to 
elegantly add methods that are valid only for some types of locks, e.g. `new 
WakeLock(cpu).dimScreen()` is nonsense, but would valid from perspective of 
WebIDL and JS prototypes.

-- 
regards, Kornel

Re: [whatwg] Proposal: toDataURL “image/png” compression control

2014-05-29 Thread Kornel Lesiński

On 29.05.2014, at 23:19, Glenn Maynard gl...@zewt.org wrote:
 
 Anyway, this has derailed the thread.  We have an API for compression
 already.  It already supports a compression level argument for JPEG.
 Having an equivalent argument for PNG is a no-brainer.  The only
 difference to JPEG is that it should be described as the compression
 level rather than quality level, since with PNG it has no effect on
 quality, only the file size and time it takes to compress.

I don't think it's a no-brainer. There are several ways it could be interpreted:


1. As zlib's compression level

However, this has marginal utility, because these days even the maximum level, 
even on mobile devices, is reasonably fast. Lower level would be useful only 
for very large images on very slow devices, but UAs can have a good heuristic 
for ensuring reasonable compression time without any input from the page's 
author.

I expect exponential increase in computing power make this setting completely 
irrelevant by the time it's implemented in most browsers.


2. Enable brute-force search for best combinations of zlib's compression level, 
memory level and window size 

OptiPNG and pngcrush show that maximum settings in zlib don't always give 
smallest file and best compression is obtained by trying hundreds of 
combinations of zlib parameters.

If browsers choose this approach for a high compression level that will be a 
couple of *orders of magnitude* slower than the first option. If different 
vendors don't agree on orders of magnitude of time it takes to compress an 
image, such parameter could be unusable.


3. Compression parameters in other gzip implementations

For example Zopfli compressor produces files smaller than zlib, but is much 
much slower. Instead of 1-9 scale it takes number of iterations as the 
compression level.

And it can even use a totally different approach to compression level: I've 
modified Zopfli[1] to make it aim for constant processing time on any machine. 
Faster machines will just produce smaller files. Browsers could use this 
approach to ensure every PNG is compressed in  0.5s or so, or the compression 
level parameter could be a number of seconds to spend on the compression.


And that's just for lossless PNG. It's possible to encode standard PNG in a 
*lossy* fashion (http://pngmini.com/lossypng.html), and there are few ways to 
do it:

Images can be converted to PNG-8 (vector quantization is a form of lossy 
compression) and then compression level could be interpreted as number of 
unique colors or mean square error of the quantized image (the latter option is 
used by http://pngquant.org). This generally makes files 3-4 times smaller, but 
has a limit on maximum quality that can be achieved.

For higher quality it's possible to make truecolor PNG lossy by taking 
advantage of the fact that PNG filters are predictors. Instead of writing all 
pixels as they are in the input image the encoder can replace some pixels with 
values matching filters' prediction. This simplifies the data and generally 
halves the file size (and costs almost no extra CPU time). The threshold used 
to choose between source and predicted values for pixels acts similarly to 
JPEG's quality level.

So there are multiple ways such parameter can be interpreted, and it can result 
in wildly different visual quality, file size and time taken to compress the 
image.

-- 
regards, Kornel

[1] https://github.com/pornel/zopfli

Re: [whatwg] Simplified picture element draft

2014-01-07 Thread Kornel Lesiński


On Sat, 04 Jan 2014 06:36:27 -, Adam Barth w...@adambarth.com wrote:


In order for the HTMLPreloadScanner to issue preload requests for
picture elements, the HTMLPreloadScanner would need to be able to
evaluate arbitrary media requests.  That's difficult to do without
joining the main thread because the media query engine works only on
the main thread.


Solution I've suggested originally was that when the selection algorithm  
encounters a media query it cannot evaluate yet, it aborts selection,  
waits until conditions change and retries selection form the beginning.


This means that:

 * all images that can be selected by the preloader will be selected, and  
they'll be selected as soon as it is possible,

 * browser will never load any irrelevant image,
 * browsers can optimize when and which MQs match without affecting  
correctness.



For example:

   picture
   source media=resolution:2dppx src=first
   source media=max-width:px src=second
   source media=super-complex-media-query-to-evaluate:  
on-main-thread-only src=third

   /picture

in an iframe without layout.

1. If you know resolution and the first MQ matches, then load src=first  
immediately. Done!
2. If you don't know viewport size then wait until any conditions change  
(i.e. either viewport size becomes known OR resolution changes) and go to  
step 1.
3. If the second MQ matches then load src=second immediately (might still  
happen in the preloader). Done!
4. If you can't evaluate complex MQ in the preloader, then wait until  
control goes back to the main thread and go to step 1.


By wait here I mean the selection algorithm is deferred for the given  
picture only, and nothing else is blocked.



Equivalent of it in the current spec would be something like this:

Before step 7 in http://picture.responsiveimages.org/#update-source-sets  
add:


6b. If child has a media attribute, and its value is a valid media query  
which UA temporarily cannot evaluate then exit this sub-algorithm and  
/select an image source/ again after a UA-specific delay.



temporarily cannot evaluate is completely up to UA. It may mean unknown  
sizes in iframes, it may mean non-trivial queries in the preloader, etc.


UA-specific delay could be waiting for any media query in the picture  
to change, or it could simply mean ignoring the picture in the preloader  
and doing evaluation properly on the main thread/when layout is  
calculated, etc.


exit this sub-algorithm will either cause an earlier source that has  
unambiguously matched to be loaded or empty source set will cause  
selection algorithm to do nothing.


--
regards, Kornel

[whatwg] NodeList.forEach/map/filter still doesn't work

2014-01-04 Thread Kornel Lesiński



Everywhere on the web where NodeList.forEach() is mentioned, everybody  
agrees that's something that is expected to work, but doesn't. It's  
followed by a list of excuses why it doesn't work, as if it was a  
completely intractable problem that nobody can ever fix in any way  
whatsoever.


Can we please switch efforts from explaining why it's broken to actually  
fixing it?



It's sad that I can't use  
document.querySelector().filter().map().forEach() without patching  
prototypes myself. ES6 Array.from(), even with syntactic sugar, is a  
band-aid. NodeList.forEach() still doesn't work, but should.



I don't think anybody cares for NodeList.forEach/map/filter/etc to be  
real Array functions, so I'd love to see even a simplest fix like:


NodeList.prototype.map = function(...whatever) {
return Array.from(this).map(...whatever);
}

NodeList.prototype.forEach = function(...whatever) {
return Array.from(this).forEach(...whatever);
}

etc.

--
regards, Kornel Lesiński

Re: [whatwg] Styling form controls (Was: Re: Forms-related feedback)

2013-12-05 Thread Kornel Lesiński

On Wed, 04 Dec 2013 16:12:50 -, TJ VanToll tj.vant...@gmail.com  
wrote:


The datepicker also shows the problem that using pseudo-elements as  
styling hooks presents. The calendars presented on mobile browsers and  
desktop browsers are radically different. Even if you wanted to  
standardize certain hooks, there is literally nothing in common across  
the implementations.


Maybe instead of coming up with one set of pseudo-elements that's limited  
to the lowest common denominator we should have multiple completely  
different sets of pseudo-elements for each kind of interface?


input::calendar.month-view-grid ::first-week-row {...} // typical desktop  
style


input::calendar.spin-wheel ::month-spinner {...} // iOS style

(or any other syntax with cats/hats/dogs/pseudo-functions, as long as it  
groups pseudo-elements per kind of calendar UI)


This way developers assuming date pickers are grids with a month view  
could style specific pseudo-elements for this layout and mobile browsers  
could ignore these styles completely.



--
regards, Kornel

Re: [whatwg] Simplified picture element draft

2013-11-27 Thread Kornel Lesiński


On Wed, 27 Nov 2013 00:48:56 -, Simon Pieters sim...@opera.com wrote:

You introduce a proxy that needs to be tested to see that it works in  
different scenarios (e.g. removing an attribute, that events are  
forwarded properly, that it does not affect parts it shouldn't like  
document.images, that the context menu works, etc.).


You introduce a (or two) new fallback mechanism.

You haven't specified that picture should be able to be drawn on a  
canvas in 2d (and WebGL?).


Thanks, very good examples. Now I understand (although I wish specifying  
it exactly like img would make that easy enough).


--
regards, Kornel

Re: [whatwg] Simplified picture element draft

2013-11-25 Thread Kornel Lesiński


On 25 November 2013 08:00:10 Yoav Weiss y...@yoav.ws wrote:


It contains some parts that I'm not sure have a consensus around them yet:
* It defines picture as controlling img, where earlier on this list we
discussed mostly the opposite (img querying its parent picture, if one
exists)


Controlling image is a great idea. It greatly simplifies the spec and 
hopefully implementations as well.


I chose not to expose that implementation detail, assuming that one day 
(when all UAs, crawlers implement it) we will not need explicit img 
fallback any more.


If picture was explicitly controlled by img then websites could start 
depending on that behavior, and we'd be stuck with it. OTOH picture can 
have native DOM interface and still reuse img for implementation.



* It defines img as a part of picture's shadow DOM, which we need to
see how it fits with having fallback img elements (which are necessary in
the near future).


I've added section about preloader. The img in fallback content should be 
ignored by the preloader. It's purely for picture-less UAs.


I do wonder however if fallback img should be used as equivalent of a 
source to save authors a bit of repetition. (in selection algorithm the 
first step would be for each source or img child...) or perhaps be used 
as last-resort fallback when no source matches (step 2 of the algorithm).



This proposal does contain srcset as a subcomponent, but it's not the same
srcset as defined in the HTML spec, but a modified version based on
improvements from the src-N spec. (that cover the variable-width images
use-case)


Indeed. This part of the spec isn't ironed out yet.


The proposal will also require some changes to img and specifically, when
not created by JS, img will have to avoid loading of resources until the
element is added to the DOM, and can see if its direct parent is picture.
If the parent is picture, img would then query the parent (or wait to
be controlled by its parent), otherwise, it'll load its resources as
usual.


I've specified something like that. I think it can be as simple as a flag 
that preload scanner uses internally.


I think we don't need to add any runtime behavior changes for this, as 
scripts constructing picture will not insert explicit fallback img node 
- it makes more sense to rely on picture polyfill instead (that will use 
img with correct src from the start).


--
regards, Kornel

Re: [whatwg] Simplified picture element draft

2013-11-25 Thread Kornel Lesiński


On 25 November 2013 10:59:15 Yoav Weiss y...@yoav.ws wrote:

On Mon, Nov 25, 2013 at 11:32 AM, Kornel Lesiński kor...@geekhood.netwrote:



 If picture was explicitly controlled by img then websites could start
 depending on that behavior, and we'd be stuck with it. OTOH picture can
 have native DOM interface and still reuse img for implementation.

I believe these interfaces would be something you'd need to test, so you
would have testing duplication, even if you save code duplication.


Yes, you need to test the integration point, but you only need to test that 
assignment of one attribute affects the other. You don't need to repeat 
tests that test it deeper.



 I do wonder however if fallback img should be used as equivalent of a
 source to save authors a bit of repetition. (in selection algorithm the
 first step would be for each source or img child...) or perhaps be used
 as last-resort fallback when no source matches (step 2 of the algorithm).


I agree that it would make sense for authors.


Which variant you think is better?


 I've specified something like that. I think it can be as simple as a flag
 that preload scanner uses internally.


Again, this is an issue with HTMLImageElement itself, not the preload
scanner. It'd probably require modifications to the img section of the
HTML spec.


I believe it won't be an issue in the approach I've specified - when the 
fallback img is separate from controlling image.


Scripts can avoid creating fallback img at all, because when scripting is 
enabled they will use polyfill and can treat all UAs as supporting picture. 
In that case fallback img would be like document.write(noscript) ;)


Maybe the spec should have authoring guidelines for this?

The controlling image starts with no src, so it won't download anything 
that wasn't deliberately chosen through picture.


--
regards, Kornel

Re: [whatwg] Simplified picture element draft

2013-11-25 Thread Kornel Lesiński



The advantage of the scheme that zcorpan proposed is that there is no magic 
proxy; we just add a capability to img to select its source using more 
than just a src attribute. This has better fallback than your design and is 
easier to implement.


I believe that from testing perspective both approaches are equivalent.

The spec I propose *is* only another way to control src of an image.

The only difference is that I don't expose the img to scripts.

That may make it even simpler, because you can't have odd cases like author 
moving/removing the controlling img or setting values directly on img that 
conflict with picture's definitions.


--
regards, Kornel

[whatwg] Simplified picture element draft

2013-11-23 Thread Kornel Lesiński



I've written down proposal for the simplified source selection algorithm:

http://geekhood.net/picture-element.html

This also includes variant of the idea from the recent picture redux  
proposal to use an actual img element as the basis for the picture  
element definition.


This draft doesn't include all features of src-N *yet*, but I expect this  
to be added either via extended srcset syntax to something like source  
sizes once there's consensus how to approach this.


To simplify implementation even further I've allowed UAs to flatten  
fallback DOM to a plaintext string (in case they need to emulate img alt  
for existing screen readers or accessibility APIs).


I've dropped usemap. It could be added, but I'm not sure if there is need  
for it.


I've specified very few IDL attributes. This area may need to be extended.

--
regards, Kornel

Re: [whatwg] The src-N proposal

On Wed, 20 Nov 2013 05:24:21 -, Bruno Racineux br...@hexanet.net  
wrote:



If your sources and breakpoints are hard-coded in your articles (stored
DB), and you suddenly have to change your site's theme, or add a new  
image at the platform level or a new resolution? What if one breakpoint  
is no

longer relevant? Or what if you change designs with a complete new
responsive approach? How does an inline syntax help me with that case?

You can be stuck. That forces you to regenerate all the img src(s) of
your articles with your new layout and new inline breakpoints.


I sympathize with the problem. Unfortunately we have a hard requirement of  
supporting the preload scanner, which means we absolutely cannot wait for  
any external file.


And since we can't wait for any external file, we can't wait for  
stylesheets or any reusable centralized definition of breakpoints.


When HTTP/2 Push becomes a standard feature preload scanner won't be so  
important any more and we'll be able to revisit this.



A centralized css-subset approach do not have such difficult problems.
Verbose aside, to me this all screams: RespIMGs has to be a CSS related
feature with centralization of custom MQs and srcset(s) at the head.


With preload scanner limitation definitions in head is the best we could  
possibly do. I have proposed Media Query Variables intended to be used in  
style in head for responsive images.


I've also wanted MQ variables to be usable in external stylesheets to  
reduce repetition in regular @media CSS, but even mere possibility of  
authors misusing external CSS definitions for responsive images (which  
would achieve centralization you want, but also get in the way of preload  
scanner) made browser vendors feel uneasy about this proposal. I hope to  
convince them otherwise, but until then your best bet is to use  
server-side templating language (or project-wide find and replace) to  
define your breakpoints once.


--
regards, Kornel

Re: [whatwg] The src-N proposal


On Tue, 19 Nov 2013 22:07:33 -, Simon Pieters sim...@opera.com wrote:

In http://lists.w3.org/Archives/Public/public-respimg/2013Oct/0045.html  
I discuss a problem that a new element would have, namely that it would  
require a new fallback mechanism and a lot of stuff would need to be  
duplicated from img.


Do we need usemap? We can probably drop it. We don't need to replicate  
lots of legacy features and quirks of img.


I think the upside is that we can ship picture with almost no features,  
and re-add them only as necessary.



For the fallback:

canvas is an existing example of a picture with a fallback DOM, so  
browser vendors already have to implement/implemented fallback for  
picture-like element.


I would go further and simplify it by forbidding all interactive  
(focusable) elements in picture fallback DOM. Canvas already forbids  
interactive elements with some exceptions, but for picture we don't even  
need these exceptions. This authoring rule can be validated easily, and  
allows UAs to avoid real difficulty of handling focus in fallback.


To make picture easy to plug into existing ATs I suggest specifying that  
UAs MAY interpret fallback content as text extracted using innerText  
algorithm (preserves space between elements) with additional rule that  
@alt from any img in the fallback is extracted as well (so pictureimg  
alt=old alt/picture as well as picturepfancy alt/p/picture  
will have good accessibility in all UAs).


This should be zero extra work for implementors, since that's what they  
already do for copying selection to plain text clipboard.


With plain text extracted from the fallback it will be possible to reuse  
accessibility interfaces designed for img alt.


When picture implementations mature we may eventually be able to let  
authors rely on more structured fallback. In any case we're better off  
than with strictly-plaintext-forever img alt, and the first version of  
picture can be guaranteed to be be easily implementable in terms of  
img.


At this point we could change the name of the wrapping element to  
picture and basically have the same syntax as current picture except  
there would be a required img child element.


The x-picture polyfill implements picture using img  
(http://uniqname.github.io/x-picture/), so that's definitely a way to do  
simple implementation.


An img element will be de-facto required for a while as a fallback, but  
could it be optional eventually? I think that even if browsers implement  
picture using img, the img element itself should be hidden in shadow  
DOM.



If we don't explicitly define picture as wrapper for img then yes,  
we'll need separate test cases for picture, but:


- hopefully plenty of cases can be adapted with little more than  
find'n'replace img src= with picturesource src=
- We don't need to bring all the legacy baggage of img, so a bunch of  
tests for Netscape'isms can be deleted.
- Image element has weird stuff like .complete property that can change  
synchronously. Kill it! With clean slate we can define only minimal,  
quirk-free API that is much easier to deal with.
- Test cases is something that can be shared between browser vendors, and  
the community can help adapt img test cases to picture, so we can  
spread the effort.


--
regards, Kornel

Re: [whatwg] The src-N proposal

 than
complicating microsyntax with attribute-within-attribute and/or extra
layers of delimiters and escaping).

Authors already have ways of dealing with verbosity of HTML and DOM APIs
(templating, jQuery, etc.), and we have proposals for reducing repetition
with orthogonal features like Media Query Variables, so I think we can
afford starting with a bit verbose, but sane and straightforward syntax.

The lesson we learnt from video/source isn't that the pattern is an 
easy choice. It's that we should avoid it if at all possible. :-)


video was undoubtedly painful, but I've looked at video test cases and
media selection algorithm and I think the pain was caused by
video-specific problems and complexity of MediaElement algorithms and
APIs, and is not inherent to use of elements in HTML in general.

Images don't need to expose API for buffering, seeking, playback states,
etc. Image sources can be evaluated using simple, stateless atomic
algorithm - basically same algorithm as you'd use for an attribute, but
instead of using custom attribute parser you read attributes from child
nodes.

--
regards, Kornel

Re: [whatwg] picture redux

On Wed, 20 Nov 2013 17:25:07 -, Tab Atkins Jr. jackalm...@gmail.com  
wrote:



Simon Pieters wrote up Kornel's earlier approach to a saner, more
palatable source selection algorithm for picture (rather than
copying video/audio).  This approach also has a new wrinkle:
picture *requires* an img child, and it's the img that still
actually displays the image.  The picture element is just a wrapper
for the img+source elements, and provides a context for the source
selection algorithm.  This makes testing substantially easier, as we
can limit ourselves to testing the source selection algorithm, and
probably makes implementation easier as well.


Can we hide the controlling img in shadow DOM? And make  
HTMLPictureElement the interface that proxies relevant properties/events  
to the internal img?


Reuse of img is a great idea for simpler implementation and testing, but  
maybe we don't even need to expose that fact to the authors.


--
regards, Kornel

Re: [whatwg] The src-N proposal

2013-11-18 Thread Kornel Lesiński

On Mon, 18 Nov 2013 16:47:08 -, James Graham ja...@hoppipolla.co.uk  
wrote:



On 18/11/13 16:36, matmarquis.com wrote:

I recall that some of the more
specific resistance was due to the complication involved in
implementing and testing existing media elements, but I can’t claim
to understand precisely what manner of browser-internal complications
`source` elements brought to the table.


The fundamental issue is atomicity; setting one or N attributes is an  
atomic operation from the point of view of script; creating N elements  
is not. This creates complexity because the algorithm has to deal with  
the possibility of DOM mutation changing the set of available sources  
before it has selected the correct one. I believe there was a proposal  
that simplified the semantics by ignoring mutations, but I hear it ran  
into problems with animated images, which I haven't understood in detail.


I agree that source as specified for video and initially for picture  
was a mess, but that doesn't have to be the case. The complexity was  
mainly caused by stateful algorithm exposed to JS, which is not necessary  
for picture.



It's *is* possible to have use N elements atomically.


I've specified a simplified source selection algorithm[1] that achieves  
this. It is atomic from JS perspective.


Atomicity is achieved by always scheduling the selection algorithm to run  
on next tick (event loop spin) after mutation. This way JS can perform  
several mutations in a row without worrying about race conditions.


The algorithm I've specified is also stateless and works correctly with  
incomplete data (e.g. if packet boundary happens to be inside picture).


AFAIK it makes it as easy to implement and as safe to use as src-N.

Simon, who initially raised concerns about use of source in picture  
found that solution acceptable[2].


I'd love to hear feedback about simplified, atomic source from other  
vendors.



[1]  
https://github.com/ResponsiveImagesCG/picture-element/issues/62#issuecomment-24479164

[2] http://lists.w3.org/Archives/Public/public-html/2013Sep/0185.html

--
regards, Kornel

Re: [whatwg] The src-N proposal

2013-11-18 Thread Kornel Lesiński


On Tue, 19 Nov 2013 01:12:12 -, Tab Atkins Jr. jackalm...@gmail.com
wrote:


AFAIK it makes it as easy to implement and as safe to use as src-N.

Simon, who initially raised concerns about use of source in picture
found that solution acceptable[2].

I'd love to hear feedback about simplified, atomic source from other
vendors.


The cost there is that picturesource is now treated substantially
differently than videosource, despite sharing a name.


The substantial difference is that it lacks JS API exposing  
network/buffering state, but IHMO that's not a big loss, as those concepts  
are not as needed for pictures.


IMHO the important thing is that on the surface (syntactical level)  
they're the same - multiple source elements where the first one matches.



Otherwise, though, I'm fine with this as well.  The only innovation
that src-N offers over picture is the variable-width images syntax,
and that can be baked into source src as well.


That was exactly my thought. Combination of src-N features with less  
contentious syntax would be ideal.


source can support number of attributes, so if there are objections to  
some features or parts of src-N syntax, it can be split into multiple  
attributes on source to be introduced gradually later/as needed (e.g.  
source media, source sizes, source 3d-google-glass-hologram-set,  
etc.) without risking explosive complexity of combined microsyntaxes.


--
regards, Kornel

Re: [whatwg] The src-N proposal

2013-11-18 Thread Kornel Lesiński

On Mon, 18 Nov 2013 23:18:37 -, Bruno Racineux br...@hexanet.net  
wrote:


All I hear from implementors as a whole, is that: you don't want to go  
the css imgset or image-set road, you won't use src-templates, and you  
don't

want any new macro. Seriously, what it left?


Indeed, the discussions are difficult, but hopefully we're making progress.

For all it's worth, my outside take on both of srcset and src-N has  
always been that it's not DRY enough, and more unnecessary bloat to  
pages, due

the long unnecessary repetition of img-path(s) for each img of similar
size, repeating the same pattern over and over for image galleries, and
lack of src-template (or regex pattern) approach to this problem.


I agree that none of current proposals is perfect and all have degree of  
repetition and verbosity.


However, the most terse syntaxes are starting to look like Perl. It's not  
always the best idea to squeeze every byte out of a syntax.


Even if none of existing proposals is perfect in terms of DRY, I think  
overall they're good enough to be useful. I'm not concerned about  
verbosity, because gzip is excellent at removing cost of any repetition,  
so on the wire the most verbose and the most terse syntax cost the same.  
In terms of memory footprint we're talking about few attributes or  
elements that take bytes/1-digit kilobytes... while displaying megabytes  
of high-DPI RGBA bitmaps.


We should be able to add URL templates or another DRYing method later  
(especially to source which can take additional attributes easily  
without complicating syntax), and such layering/decoupling may actually be  
a more elegant architecture.


I would consider src-N more friendly, with perhaps a new 'base src in  
the head dedicated to src-N(s) and, proceed to includes custom MQs in  
the

head at the same time (which is inline css in the head anyway), to a
least reduce some of its verbosity...


As you know there has been proposal for Media Query Variables, so it seems  
quite probable that a similar thing can be added for other properties of  
responsive images as well.


One way to convince browser vendors that such syntax is needed is to let  
them ship the basic version with full URLs, and then you'll have proof  
that URL patterns emerge and authors complain about verbosity (or not :)



Either way, it's quite pathetic to watch implementors argue over two half
baked quite verbose solutions, from a distance, after nearly 3 years
thinking of this... Even worse, suggesting to go ahead with something  
incomplete,

not knowing what the future completion will actually consist of.


The issues and ideas discussed here look a lot like discussions in RICG  
years ago, so hopefully we'll eventually come to the same conclusions as  
RICG did ;)


--
regards, Kornel

Re: [whatwg] The src-N proposal

2013-11-16 Thread Kornel Lesiński


On Sun, 10 Nov 2013 08:20:33 -, Adam Barth w...@adambarth.com wrote:

This is similar to AppCache vs Alex's ServiceWorkers. AppCache  
addresses a small set of use cases, probably not enough. ServiceWorkers  
provides the
tools to address a lot of use cases, but isn't directly itself a  
solution;

you use it to build solutions. Another example would be the WebForms2
repetition model, vs Rafael's template. The repetition model idea  
solved some specific use cases, but trying to make it solve all use  
cases would

be a hugely complicated endeavour and would be really ugly. template
provides a tool with which you can build specific solutions, but isn't
itself a direct solution.


I basically agree with Ian.  Let's address the simple use cases first
(i.e., device-pixel-ratio switching) and worry about the more complex
use cases in the future.


If we go that path I'm afraid we'll end up with a horrible mess of several  
incomplete client-side and server-side solutions clobbered together with  
preloader-killing scripts.


The closest thing to what Ian is suggesting is x-picture implemented  
with img postpone, but due to standardization failure it won't be able  
to benefit from image preloader or offer users/UAs ability to control  
image selection.


Basically authors will hate us. We've been going in circles for a couple  
of years now and all we have to offer is an incomplete solution? And  
browser vendors can't even agree which one of the half-baked solutions is  
it going to be :(


--
regards, Kornel

Re: [whatwg] The src-N proposal

2013-11-08 Thread Kornel Lesiński


* The developer community and the RICG are rallying behind src-n, with work
on picture being discontinued in favor of src-N.


I'd like to clarify that src-N got support from RCIG on assumption that 
picture has been rejected by browser vendors and has no future.
However, many members have expressed that they prefer picture syntax over 
src-N.


--
regards, Kornel

Re: [whatwg] High-density canvases

2013-09-10 Thread Kornel Lesiński


On Tue, 10 Sep 2013 21:22:51 +0100, Dean Jackson d...@apple.com wrote:


I think there are two separate things a developer might want:

- the number of actual pixels that correspond to 1 CSS px without zoom
- the page zoom

If you merge the two, then an unsuspecting developer might think that  
the user has zoomed in by 2x on an iPhone, and decide to make things  
smaller.


Do you have an example of a page that does make things smaller to counter  
the zoom? Are you referring to some iPhone-specific workarounds (like  
position:fixed elements being problematic for zoom?)


I assumed that sites which don't like being zoomed in would just block it  
via meta viewport.


--
regards, Kornel

Re: [whatwg] Blurry lines in 2D Canvas (and SVG)

2013-07-23 Thread Kornel Lesiński

On Wed, 24 Jul 2013 01:18:35 +0100, David Dailey  
ddai...@zoominternet.net wrote:



Just affirming what you've said in SVG:
http://cs.sru.edu/~ddailey/svg/edgeblurs.svg

The middle rects are crisp, having been merely translated leftward and
downward by half a pixel. Zooming in from the browser rectifies the  
problem

(as expected) after a single tick.

I remember folks discussing sub-pixel antialiasing quite a bit on the SVG
lists circa fall/winter 2011. It seemed to cause some troubles for D3. Is
that the same issue?


It's not a bug, it's a feature ;)

The line is centered around edge of the box. You haven't specified whether  
you want the line to be outside or inside the box (or overlapping left  
edge of the box, but not the right, etc.), so you get line in the middle  
approximated as well as possible.


It's not intuitive. It's a pretty common pitfall, but it's logical.

For 1-pixel lines it could be fixed by allowing authors to specify that  
path should be stroked with lines aligned to inside/outside of the path  
(which is a useful feature on its own).


--
regards, Kornel

Re: [whatwg] Script preloading, ES6 modules

2013-07-15 Thread Kornel Lesiński



ES6 modules[1] have a script loader API[2].

That API is pretty powerful to the point it can emulate other script  
loaders, load files that are not ES6 modules, and even load text files  
that aren't JS (intended for compilation of coffeescript-like languages,  
but could be abused for anything):


https://gist.github.com/wycats/51c96e3adcdb3a68cbc3#using-existing-libraries-as-modules


There's a very high overlap between module dependencies and script  
dependencies proposal. I think at very least it would be useful to define  
script dependencies in terms of ES6 modules, or even abandon markup  
solution to avoid duplicating features.



ES6 modules however do not solve the performance problem. In fact they  
would benefit from UA having a list of all dependencies up front  
(otherwise file's dependencies can only be discovered after that file is  
loaded, which costs as many RTTs as the height of the dependency tree).


So I think that eventually ES6 modules + link[rel=subresource] could be  
the answer. The link would expose URLs to (pre)load for performance, but  
modules would handle actual loading/execution for flexibility and  
reliability.


--
regards, Kornel

[1] http://wiki.ecmascript.org/doku.php?id=harmony:modules
[2] http://wiki.ecmascript.org/doku.php?id=harmony:module_loaders

Re: [whatwg] Script preloading, optional dependencies

2013-07-14 Thread Kornel Lesiński


On Fri, 12 Jul 2013 21:20:57 +0100, Kyle Simpson get...@gmail.com wrote:

So, just to clarify, `script dependencies=…` waiting on some other  
script tag is ONLY waiting on that script tag loading to have some  
sort of positive network result, whether that be a 2xx, 3xx, 4xx, or  
5xx, and it cares not whether the script in question actually loaded,  
nor whether it fired its `onerror` event? Do I have that correct?


What should happen is an interesting question.

If execution continued when dependency fails to load/parse, then later  
scripts could try to recover/use fallback/run with reduced functionality  
(e.g. if jQuery plugin for fancy animations fails to load the application  
can disable fancy animations and run anyway).


OTOH aborting execution of the rest of the dependency chain is probably  
what most developers expect, and it often makes sense as most scripts  
without dependencies met will fail.



So maybe a concept of optional dependency would be useful?

e.g. error handler could signal somehow that execution should continue:

script id=fancy-animations src=//flaky.cdn/js onerror=return  
false; /


or there could be a microsyntax in dependencies:

script dependencies=fancy-animations? required-stuff /

or:

script id=fancy-animations-cdn src=//flaky.cdn/js /
script id=fancy-animations-local src=/fancy-animations-copy.js /
script id=nothing /
script dependencies=(fancy-animations-cdn or fancy-animations-local  
or nothing) required-stuff /


--
regards, Kornel

Re: [whatwg] Script preloading, non-script dependencies

2013-07-14 Thread Kornel Lesiński


On Tue, 09 Jul 2013 20:39:45 +0100, Ian Hickson i...@hixie.ch wrote:


Would something like this, based on proposals from a variety of people in
the past, work for your needs?

1. Add a dependencies attribute to script that can point to other
   scripts to indicate that execution of this script should be delayed
   until all other scripts that are (a) earlier in the tree order and (b)
   identified by this attribute have executed.

 script id=jquery src=jquery.js async/script
 script id=shims src=shims.js async/script
 script dependencies=shims jquery src=myscript.js  
async/script


On a basic level it's similar to JS async module definition pattern  
(https://github.com/amdjs/amdjs-api/wiki/AMD), which is good, but may be  
insufficient.


require.js (AMD implementation) has useful concept of loader plugins for  
dependencies (https://github.com/millermedeiros/requirejs-plugins).
A plugin is basically a function that is told to load a path and calls  
callback when it's done.


It solves two problems:

1. not all dependencies are JS files, e.g. authors use plugins to load  
template files, JSON, images, etc.


script dependencies=load_template('view.hbs') src=view.js
script
function load_template(url, callback) {
fetch(url).then(callback);  // in require.js result is  
directly passed to the module.

}
/script

2. not all dependencies are usefully satisfied immediately after their JS  
file is loaded, e.g. some libraries may need asynchronous initialization.  
In require.js it's possible to wrap initialization in a plugin that will  
wait until it's done, so modules dependent on it can start using  
initialized library right away.


script id=load-library src=library.js /
script dependencies=load-library
function library_initialized(callback) {
$library.on('ready', callback)
}
/script
script dependencies=library_initialized() src=use-it-now.js/


Another common kind of dependency scripts have is presence of certain  
element in the DOM, e.g. `dropdown-menu.js` may require `nav id=menu`  
to be in the document _and_ have its content fully parsed before the  
script can run.


So, could script dependencies point to non-script elements?

script dependencies=menu src=dropdown-menu.js/
nav id=menu
!-- script not run yet --
ul.../ul
/nav
!-- script run at this point --

It would be nice if the browser also deferred rendering of the element  
until scripts that depend on it are run to avoid Flash of Unbehaviored  
Content (e.g. elements listed in dependencies have display:none until  
scripts are run).


--
regards, Kornel

Re: [whatwg] Forcing orientation in content

2013-07-14 Thread Kornel Lesiński

On Sat, 13 Jul 2013 08:13:03 +0100, Tobie Langel tobie.lan...@gmail.com  
wrote:


It is not uncommon for mobile experiences to rely on the accelerometer  
as an input mechanism, for example to control page scrolling (e.g.  
Instapaper) or for gameplay.


In such cases, auto-rotation of the viewport is completely disruptive to  
the user's experience and needs to be inhibited.


Indeed, this ruins accelerometer-based games.

It's also slightly problematic in applications using compass (augumented  
reality or navigation apps pointing user towards a direction) -  
auto-rotation misfires when person rotates themselves while holding phone  
in front of them.



Inhibiting auto-rotation may be sufficient, and shouldn't be too annoying.  
Browsers might even have option to unlock rotation (e.g. Instapaper shows  
rotation lock switch when you shake the device).


I suspect that games designed for being locked in a particular screen  
orientation will be forcing users to rotate device to desired orientation  
first (e.g. I can imagine racing games to refuse to start the race until  
user rotates device to landscape) — but maybe that's a good thing?



Since specific, locked screen orientation is mostly needed in games, and  
forced rotation is disruptive to other things on the screen (e.g. moving  
buttons/addressbar to other physical edge of the screen), maybe it should  
be tied to the Fullscreen API?



element.requestFullscreen({orientation:'landscape',  
autorotation:false})



--
regards, Kornel

Re: [whatwg] Script preloading

2013-07-10 Thread Kornel Lesiński


On Wed, 10 Jul 2013 16:39:42 +0100, Kyle Simpson get...@gmail.com wrote:


I personally don't care about scripts being discoverable by pre-parsers.

[...]
For instance, I've added like link rel=prefetch annotations for my  
scripts into the head of my document, and then done my normal  
script-based script loading as usual, and benchmarked if them being in  
the markup somehow magically sped up the page. I saw no appreciable  
increase in average page load speed in my testing.


The result you've seen is expected, as pre-parser and link prefetching are  
different mechanisms.


Prefetching of resources from rel=prefetch is only done after page  
completely finishes loading, so it won't affect loading time of page that  
contains it (it's for future navigation, like step1.html containing  
prefetch for step2.html).


There's rel=subresource that's for immediate prefetching in the way  
you'd expect. You should see some improvement with it in cases where  
script loader makes it impossible for browser to start fetching scripts  
before the script loader is run:

http://www.chromium.org/spdy/link-headers-and-server-hint/link-rel-subresource

HTML pre-parser is basically equivalent of having rel=subresource for  
all script elements.


--
regards, Kornel

Re: [whatwg] Adaptive Image Element Proposal

2012-09-05 Thread Kornel Lesiński

On Tue, 04 Sep 2012 22:53:57 +0100, Leif Halvard Silli  
xn--mlform-iua@målform.no wrote:



If we say that picture should have img role, then we imply that
alternative text should be provided via an attribute.


Why?


Because that is what ARIA 1.0 says about the img role: In order for
elements with a role of img be perceivable, authors
SHOULD provide alternative text or a label determined by the accessible
name calculation.


The accessible name calculation algorithm includes a clause:

Otherwise, if the attributes checked in rules A and B didn't provide  
results, text is collected from descendant content if the current  
element's role allows Name From: contents.


so if I understand this correctly, the picture element can be defined to  
allow Name From: contents, and then it will work fine as role=img and  
structured alternative content in the element.



You mean, treat img’s @alt like the caption of table, for
instance? That is: Make it img a required part of the picture
construct, for instance? I do think that integrating img into
picture as a part of the picture compound element, sounds
interesting. And it could be possible, I guess.


Not exactly required. Simply read alternative content from element's  
content. If you find img there, then read its alt. If you find text,  
table or something else, then read that.



One thing to think about is this: If we e.g. have a table which we
declare as presentational, then we also, per the ARIA rules, declare
the elements that are part of the table construct of that particular
table (td, tr etc) as presentational. So, if img was seen as part
of picture, then, to declare picture as presentational would also
affect the img.


Yes, I think it makes sense.

picture role=presentation/picture would be equivalent to img alt=  
(presentational image).


picture --role=img--/picture would be equivalent to img (missing  
alt).


picture --role=img--alternative/picture would be equivalent to img  
alt=alternative (alternative present).


--
regards, Kornel

Re: [whatwg] Features for responsive Web design

2012-09-05 Thread Kornel Lesiński

On Mon, 27 Aug 2012 12:05:00 +0100, Chaals McCathieNevile w...@chaals.com  
wrote:


While it's unlikely that screen resolution will go above 2x in the near  
future, should we be taking into account the zooming of specific  
elements that might result in the need for larger artwork? (take icons,  
that can scale all the way up to 512px or above)


Or outdoor screens that are 4m x 8m, carrying the same content meant for  
a TV display and a message to your mobile.


Use cases:
+ emergency information provision, where the sign is acting as a server  
providing information to all devices that can connect).
+ providing advertising, local information that can be rendered on large  
outdoor screens, tv-size screens, and for customers.


Those are fine use-cases, but I don't see how do they need anything  
special in relation to zooming/pixel density.


It seems to me that despite large physical size, outdoor screens are not  
unusual, because due to larger viewing distance the actual perceived size  
and pixel density isn't very different from normal screens.


For example the famous display ads in Piccadilly Circus in London have  
easily noticeable pixels, so they're merely a 1x screen, not even 2x  
yet.


--
regards, Kornel

Re: [whatwg] Hardware accelerated canvas

On Tue, 04 Sep 2012 19:35:32 +0100, Justin Novosad ju...@chromium.org  
wrote:


That doesn't sound too evil, but the ideal solution would be one that  
would not involve web standards at all. If there was a way of ensuring  
GPU

resource persistence on mobile platforms (swap-out resources rather than
discard them), then we would not be having this conversation. Making that
happen is a debate for a different audience. Unfortunately OS and  
graphics APIs don't evolve at whatwg pace.


Indeed.

I think it'd be ideal if browsers could hide this problem from developers  
(with command logging, snapshotting or other tricks) until improvements in  
OS/drivers/hardware make this a non-issue (e.g. if the OS can notify  
applications before gfx context is lost, then browsers could snapshot then  
and problem will be gone for good)


Until then great performance can still be achieved with some heuristics  
and accepted risk of loss, e.g. don't snapshot for 1/10th of a second  
after canvas has been cleared, don't log commands from  
requestAnimationFrame() etc.


--
regards, Kornel

Re: [whatwg] Hardware accelerated canvas


On Tue, 04 Sep 2012 17:43:11 +0100, Boris Zbarsky bzbar...@mit.edu wrote:


5)  Save command stream.
6)  Have a way for pages to explicitly snapshot a canvas.
7)  Require opt in for hardware accelerated rendering.

Any others?

Of the above, I don't think #5 and #7 are realistic, for what it's  
worth.  I haven't put enough thought into the rest yet to decide what I  
think about them.


Would a mix of #5 and snapshotting work?

1. create a (fixed-size?) append-only buffer for drawing commands,
2. log all drawing commands until the buffer is full or a  
non-cheaply-serializable command (e.g. draw of video) is executed,

3. snapshot,
4. empty the buffer
5. goto 2

That could make readbacks much less frequent. Would this still be a  
prohibitively expensive solution?


--
regards, Kornel

Re: [whatwg] Adaptive Image Element Proposal

On Tue, 04 Sep 2012 21:01:32 +0100, Adrian Roselli  
rose...@algonquinstudios.com wrote:



 So, how would the hyperlink in this example work for *all* users?

I disagree with premise of this question. I don't think it should work  
for all users.


If the link is in alternative content, then by definition it is  
intended only for users who cannot see the picture, e.g. it could be a  
link to longdesc-type of page that contains no useful information for  
sighted users.


By definition the value must be an appropriate replacement for the  
image. It does not say it is solely for users who cannot see the  
picture (4.8.1.1.1).


You're right, there is nothing saying solely. However, the spec also  
doesn't say that alt needs to be accessible when images are displayed.


Currently UAs don't make it easy to see alt when images are displayed. IE  
used to display alt as a tooltip even when images were enabled, and this  
has been called out as an error and fixed.



Section 4.8.1 starts with: the value of the alt attribute provides  
equivalent content for those who cannot process images or who have image  
loading disabled..


I think it's reasonable to flip this and interpret it as alt is not for  
those who have images enabled and process them.



The link should be inaccessible to sighted users in the same way img
alt=text is inaccessible.

[...]

There are many cases where a sighted user access @alt text. The biggest  
one is when the referenced file is missing. Other cases include  
bad/dropped connections, bad reference (404), unsupported file format,  
corrupt file, and a mean game of hide-and-seek.


But that is not the case when image is displayed.

When image is not displayed *for any reason*, then the alternative should  
be shown, and then the link would be accessible.


--
regards, Kornel

Re: [whatwg] Adaptive Image Element Proposal

On Tue, 04 Sep 2012 21:47:15 +0100, Adrian Roselli  
rose...@algonquinstudios.com wrote:


My point was more about @alt being solely for non-sighted users. I felt  
that starting off with that premise could lead to conclusions based on a  
faulty base.


Ah, sorry. Of course instead of sighted/non-sighted I should have said  
when UAs display the image/when UAs don't display the image or expose  
alternative available to the screen reader


Pedantism turned out to be useful, because when the issue is phrased this  
way I now see the problem: the image may be displayed and at the same time  
a screen reader may want to access alternative content.


This makes handling of interactive content in fallback tricky, as focus  
required for the screen reader (needing to activate link in picture)  
would differ from focus required for GUI users with images displayed  
(never focusing picture content).


The easy workaround for that may be to treat picture fallback/alt  
similarly to button content, i.e. just forbid focusable/interactive  
elements. Do any UAs allow users to access a in rendered object?


--
regards, Kornel

Re: [whatwg] Features for responsive Web design

2012-08-09 Thread Kornel Lesi��ski

On 8 sie 2012, at 12:57, Florian Rivoal flori...@opera.com wrote:

 Is there a good reason to believe that * will be something other than a
 power of two?
 
 That is, could we just optimize the *x syntax away and specify that the
 first option is 1x, the second is 2x, the third is 4x, etc.?
 
 If you look at mobile phones, there are a bunch of existing devices with
 1.5 device pixel per css pixel, and also some with 2.25, so I don't
 think we can assume only powers of 2 will be used.

Pixel-perfect design for non-integer scaling ratios is very hard. To have 
evenly thin lines (1 device pixel wide) on such screens you have to use 
fractional CSS pixel sizes, and fractions need to be different for different 
scaling ratios. 

I don't think anybody will take advantage of that. IMHO non-integer ratios are 
a mistake that can/will be corrected. 

Fractional ratios have proven to be unnecessary: on desktops 1x CSS pixel 
changed from 72dpi (CRT) to 130dpi on notebook screens, but we haven't got 
fractional scaling ratios along the way. Variability in screen sizes and actual 
DPI has been accepted. The same can happen with 1.5x-2.5x screens: pretend they 
all are 2x, vary CSS pixel width/height, accept physical size of CSS pixel will 
be slightly different.

For example the 2.25 ratio doesn't make sense to me. 12.5% increase in screen 
density is going to be imperceptible. A better solution would be to use the 
crisp 2x ratio and have bigger screen area (in CSS pixels).

For mobile browsers which have zoom it's easy as they can pretend to have 2x 
scaling ratio on a virtual viewport and resize it to screen of any other 
density.

That's what Retina MacBook Pro does with whole screen when user requests 
screen resolution that isn't 1:1 with screen pixels.

You can see from previous OS X releases that Apple has struggled with 
implementation of fractional scaling ratios for years and has given up on them.

It's going to be easier to stick to 2x screens (like most desktop monitors 
settled on ~100dpi) or use fake 2x (scaled viewport) than expect authors to 
make pixel-perfect designs for 1.25, 1.5, 1.75, 2.25, etc. 

-- 
regards, Kornel

Re: [whatwg] Features for responsive Web design

2012-08-09 Thread Kornel Lesi��ski

On 9 sie 2012, at 11:06, Nils Dagsson Moskopp n...@dieweltistgarnichtso.net 
wrote:

 I don't think anybody will take advantage of that. IMHO non-integer
 ratios are a mistake that can/will be corrected. 
 
 Limiting to powers of two because it can/will be “simpler” in this case
 not only makes the attribute harder to read – it also locks vendors in.

I'm not talking just about difficulty of specifying image scaling factor, but 
overall difficulty of developing complete page layout with all box, border and 
margin sizes in fractional CSS pixels. 

I assume that if a designer cares enough to create several pixel-perfect 
versions of images, they will also want same pixel perfection from all other 
page features, e.g. will hate that on 1.5dppx screen a 1px border is rendered 
with 1dpx in one place, but 2dpx in another (or differently blurry 2dpx in both 
places) and will want to specify 0.px or 1.px border instead. 

One stylesheet can be easily reused for   pixel-perfect 1x/2x layout, but 
pixel-perfect 1.5x requires its own sizes incompatible with 1x/2x. 

 Apart from it possibly being a self-fulfilling prophecy – isn't this
 too much premature “optimization” ?

I think we can safely assume that authors will always want to prepare as few 
assets and stylesheets as they can, and will prefer integer units to fractional 
ones (1px line vs 1.px line). 

 Btw, I am not aware of any other attribute that has a logarithmic scale
 baked in – are you?

Best scaling ratios are linear, rounded to nearest integer (i.e. 3x is OK too).
That's quite common in computer science, and not surprising when dealing with 
discrete unit like device pixel.

-- 
regards, Kornel

Re: [whatwg] The pic element

2012-06-04 Thread Kornel Lesiński

On Mon, 04 Jun 2012 01:05:23 -0500, Anselm Hannemann Web Development  
i...@anselm-hannemann.com wrote:


An alternative is to pick different delimiters. See, for instance,  
http://tools.ietf.org/html/rfc2295#section-8.3.


I also would like to see another delimiting syntax which is clearer.  
What about JSON-syntax or just  | ?
I mean a backslash is not that common in a URL but commas are more and  
more and you all know that escaping is no fun.

So we should really try to avoid this.


Another character could work in theory, but I wonder whether it would work  
in practice.


For example meta name=viewport was documented to support only comma, but  
thanks to silent error recovery authors ended up using and relying on  
semicolon:

http://lists.w3.org/Archives/Public/www-style/2011Oct/0652.html

I wonder whether reverse of it could happen with list of sources, e.g.  
unexpected comma parsed as invalid media query could end up delimiting  
sources in some implementations, and then we'll end up with worst of both  
worlds (both ambiguous comma and other unintuitive delimiter needed for  
web-compat).


--
regards, Kornel Lesiński

Re: [whatwg] The pic element

2012-06-04 Thread Kornel Lesiński

On Mon, 04 Jun 2012 00:56:55 -0500, Anselm Hannemann Web Development  
i...@anselm-hannemann.com wrote:


pic src-xs=small.jpg media-xs=(max-width:15em)  
src-xl=large.jpg alt=alt text title=title text/pic


I don't mind either way, but this seems a bit more noisier and less  
compact.


source can be an option for authors who prefer separate attributes.


I’m still digesting the overall proposal, but regarding the media-*
and src-* attributes, my main concern would be limiting ourselves by
prescribing a set number of sizes (e.g. small, medium, and
large) when in reality there may be more nuance in art direction
than that. might accommodate.

That is only partially true because this is just a proposal by me which  
can easily be changed to another not limiting syntax…
But nevertheless you might not have more than 9 different file-sizes,  
right? This is what is covered by mine (xxxs, xxs, xs, s, m, l, xl, xxl,  
xxxl).


I initially presumed that any name could be used, similarly to data-*,  
e.g. src-foobar= media-foobar=. However, that doesn't imply any  
ordering (attributes don't have order by definition), and that complicates  
source selection algorithm.


In that case I think the syntax with separated attributes is less clear  
than a microsyntax in a single attribute — list in a single attribute has  
obvious ordering and doesn't rely on an ordered set of predefined names.


--
regards, Kornel Lesiński

Re: [whatwg] The pic element

2012-06-04 Thread Kornel Lesiński

On Mon, 04 Jun 2012 01:02:55 -0500, Anselm Hannemann Web Development  
i...@anselm-hannemann.com wrote:


• Improved alternative text — allows structured fallback, avoids  
duplication.
This is where I do not agree. If you use MQ style with source you  
have a messy markup when writing alternative text inside the  
pic-element.


Since source is not read nor displayed, it doesn't matter. You can  
simply treat entire content as fallback.


Sure but why? It is much more clearly to use the alt-attribute than  
using text between container and child elements IMO.


object, iframe and canvas use content as a fallback. XHTML2 was  
supposed to use this fallback model for all elements (global src). It  
seems that img alt was just a mistake made in early development of HTML.


Since it's impossible to introduce void element at this point (XML and DTD  
boats have sailed …or sunk) we'll have to have /pic anyway. At least we  
can make the best of it and use it for rich fallback that wasn't possible  
with img before, and which is harder to ignore than an attribute,  
because the parser will require /pic.


I'm trying to avoid need for yet another opt-out from the past like  
doctype and meta charset.
It'd be great if in 10-20 years all you had to do is type pic src  
instead of img src to get first-class support for hires images.


To address Tab's concern the default is connected to image-resolution  
in CSS, so you can change it if you need to:


http://lists.w3.org/Archives/Public/public-whatwg-archive/2012May/0398.html


Yes but won't we at least dimiss img from our new code? I thought img is  
only the fallback…

And then we should always serve the minimum resolution first.
Regardless which resolution this minimal file has, it should be the @1x  
IMO.


Or am I missing something?


Yes, img is the fallback until all UAs start supporting the new element.  
I'm not sure what do you mean by we should always serve the minimum  
resolution first. If alternatives are offered to the UA, IMHO it should  
be up to UA whether it downloads low-res first or only high-res image.



I'm afraid that when high-dpi displays become the norm the minimal useful  
HTML template will grow to be something like:


!DOCTYPE html
meta charset=UTF-8
stylehtml {image-resolution:2dppx}/style

and I'm trying to find a way to avoid having all authors add yet another  
boilerplate opt-in.


I don't know what to do with default-lowres background-image, so maybe  
this is an unavoidable problem :(


--
regards, Kornel Lesiński

Re: [whatwg] The pic element

2012-06-01 Thread Kornel Lesi��ski

 not appropriate to have alt for 
non-visual agents vary based on visual properties. 

 What about 
 * usemap
 * ismap
 ?

Aren't these legacy features?

-- 
regards, Kornel

[whatwg] The pic element

2012-05-31 Thread Kornel Lesiński

 is `source` or `img` include it as an  
alternative
	2. if the child element is `noscript` and scripting is disabled,  
include all `source` and `img` children of the `noscript` as  
alternatives


e.g.

pic
   source !-- yes --
   noscriptimg/noscript !-- maybe --
   videosource/video !-- no --
/pic


The `source` element has following attributes:

* `media` — same as `media` part in `pic src`
* `resolution` — same as `resolution` part in `pic src`
* `src` — single URL without escaping or microsyntax
* `width` and `height` — analogous to `img width/height` for each  
alternative image


The `img` element in `pic` is equivalent of `source src=… 1x  
media=all`.


The resolution is interpreted as a property of the image. It describes  
ratio of image pixels to CSS pixels for image's intrinsic size, like the  
CSS `image-resolution` property.


It acts as a hint for image selection, but not as strictly as a media  
query. User-agent is free to use 2x images on 2x (Retina) displays, but  
may opt not to if network speed/cost or memory limitations prohibit this.


The user-agent may also opt to use a 2x image on a regular 1x display if  
it the page is zoomed in, when printing, etc.


Authors are encouraged to always use resolution descriptors instead of  
`device-pixel-ratio` media query to allow UAs optimize selection of image  
resolution.


Matching of media queries in `pic` must be consistent with matching of  
`@media` in CSS and `matchMedia` JS API, i.e. UA is not allowed to ignore  
or fake values in media queries to pick different image alternative  
(unless it does it consistently for the whole page at the same time).


pic src=image.jpg (min-width:500px)/pic

In the above example no image is shown (equivalent of transparent image  
with intrinsic size 0x0) if the viewport is narrower than 500px.



Algorithm for selection of the alternative:

 1. For each alternative in order
1. if media query does not match, ignore the alternative
2. if the media query matches
		1. pick all alternatives with media query identical to the one that  
matched

2. ignore all other alternatives
2. If there are no alternatives left, use transparent image with intrinsic  
size 0x0

3. Select any image among picked alternatives

This algorithm may be re-evaluated between any event loop runs.

For purpose of this algorithm media queries are identical if they parse  
to the same sequence of tokens, ignoring insignificant whitespace. The  
default/implied `all` values are identical to explicitly specified `all`.




Generally the first alternative (in `src` left to right, nested  
`source`/`img` in source order) that has media query that matches wins.


If there's more than one alternative with an identical media query, UA can  
choose among them, e.g. select best fit resolution or file format.


Note that resolution is not part of media query.
`pic src=img2 2x, img1 1x` is same as `pic src=img2 2x all, img1 1x  
all`, and since both alternatives have implied media query `all` that  
matches, UA can choose among them.



pic src=imageA (min-width:500px),
  imageB 1x (min-width:300px),
  imageC (min-width:800px),
  imageD 2x (min-width: 300px)

If `(min-width:500px)` matches, then imageA must be used regardless of  
screen density or UA preference (since it's first to match and there are  
no other alternatives with such media query).


If `(min-width:500px)` doesn't match, but `(min-width:300px)` matches,  
then UA may choose between imageB and imageD (both have the same query  
that matched).


imageC will never be used (because imageA will always match earlier and it  
has a different media query).



pic src=portraitHD.jpg 2x (orientation:portrait),
  portrait.jpg   1x (orientation:portrait),
  landscapeHD.jpg 2x,
  landscape.jpg   1xalt text/pic

UA can choose resolution of portrait or landscapa version of the image.  
Same as:


pic src=portraitHD.jpg (orientation:portrait), landscapeHD.jpg,
  portrait.jpg 1x (orientation:portrait), landscape.jpg  
1xalt text/pic



--
regards, Kornel Lesiński

Re: [whatwg] responsive images

2012-05-22 Thread Kornel Lesi��ski

On 22 maj 2012, at 05:53, Paul Court p...@pmcnetworks.co.uk wrote:

 As a HTML author and programmer, I just cannot see myself implementing the 
 current srcset proposal on sites. As a programmer, it has very much got what 
 we would call a bad code smell.
 
 img src=face-600-...@1.jpeg alt= srcset=face-600-...@1.jpeg 600w 200h 
 1x, face-600-...@2.jpeg 600w 200h 2x, face-icon.png 200w 200h
 
 Not to mention, what happens when a 3x device is released?

2x image will be used, upscaled. 

I think that 3x device is very very unlikely to ever happen, since 2x screens 
are may be dense enough to have pixels smaller than human eye can see.

 1x and 2x are. Is it 2x 72 or 2x 96? 
 and isn't 600-200@2 just the same as 1200-400@1?

'x' is not a media query, but property of the image. 

600@2 is a higher quality version of 300@1. 

 Perhaps something like this:-
 
 picture
src=some.img?{width}-{height}@{dpi}
img src=some-fallback.img
 /picture
 
 You could then define a list of parameters that the browser supports as 
 replacements. Eg. viewport-width/height, dpi, density. This could also be 
 carried over to other tags.

If width/height is a virwport size then it will generate lots of unique URLs 
(requires server to generate many images and proxies to cache them separately), 
and may generate lots of requests when window is resized. 

It doesn't work with static file servers, and may be costly/problematic for 
CDNs. 

Another risk is that authors will create files only for size/DPI of iPhone and 
iPad and never supply images for 3000 other resolutions of various Android 
devices. 

-- 
regards, Kornel

Re: [whatwg] Features for responsive Web design

2012-05-22 Thread Kornel Lesi��ski

Sorry, I forgot to clarify this ― I had in mind adding width/height on each 
source element, not on picture.

-- 
regards, Kornel


On 22 maj 2012, at 16:01, Maciej Stachowiak m...@apple.com wrote:

 
 On May 21, 2012, at 9:37 PM, Kornel Lesi��ski kor...@geekhood.net wrote:
 
 
 
 There’s no prior precedent this sort of thing―there’s no reason we can’t 
 find a way to preserve an image’s intrinsic width using `picture`. I wonder 
 if simply adding `width` and `height` attributes on the element (similar to 
 `img`) might solve this, in the event that the author wants to rely on an 
 intrinsic size instead of CSS?
 
 I think that is a very good idea. Having option to do so is good for 
 performance, as it avoids reflows.
 
 If 'width' and 'height' attributes on the picture element would do the same 
 thing as they do on img, then they would be setting the size via style, 
 rather than setting intrinsic size. Even if setting the size explicitly 
 affected intrinsic size rather than size computed via style, it would miss 
 the point of intrinsic size, which is that images get automatically the right 
 amount of space based on the image itself. Auto-sizing may not be the right 
 choice for all designs, but it is for some designs.
 
 - Maciej

Re: [whatwg] responsive images

2012-05-22 Thread Kornel Lesi��ski

On 22 maj 2012, at 15:57, Tab Atkins Jr. jackalm...@gmail.com wrote:

 On Tue, May 22, 2012 at 7:26 AM, Kornel Lesi��ski kor...@geekhood.net wrote:
 I think that 3x device is very very unlikely to ever happen, since 2x 
 screens are may be dense enough to have pixels smaller than human eye can 
 see.
 
 Tell that to printers, which can easily hit 400+dpi.  You need more
 than 2x before you make anti-aliasing fully unnecessary.

Is making AA unnecessary a goal though?

It seems to me that antialiasing isn't a complex or computationally expensive 
problem any more.

-- 
regards, Kornel

Re: [whatwg] Features for responsive Web design

2012-05-21 Thread Kornel Lesiński

On Mon, 21 May 2012 19:36:22 -0500, Mathew Marquis m...@matmarquis.com  
wrote:


picture in its current form is unable to support bandwidth-based  
negotiation well (and that's not simply a matter of adding bandwidth  
media query) and has no mechanism to specify scaling factor for  
intrinsic sizes of images.


Is there currently any documentation explaining how bandwidth is better  
handled by `srcset`? Seeing as any discussion around bandwidth is almost  
entirely theoretical, I have a hard time understanding where bandwidth  
concerns preclude one approach or the other.


No, and bandwidth proposal for srcset isn't fleshed out yet.

My — theoretical — take on this is that it's easier and safer to declare  
image sizes (e.g. in KB) and let UA figure out which ones it wants, than  
to add a bandwidth media query.


Reasons are:

 - MQ model of matching current state is not suited for variable thing  
like bandwidth.


 - MQ would require somehow explicitly quantifying bandwidth, e.g. in  
mbps, and exposing that accurately seems to me harder than having less  
exposed/less sophisticated logic in UAs like picking smallest images when  
UA is on 2G mobile network. UA could experiment with different mechanisms  
and refine their algorithms over time.



Explicit bandwidth negotiation may need to wait for green light from  
implementers. There's a risk that if initial solution is implemented  
poorly, authors will be forced to use fake/exaggerated values to get  
desired behavior, and that will poison the feature.



I hope that image resolution negotiation (1x/2x) may be a testbed for  
bandwidth negotiation. It can be safely assumed that 2x images are  
significantly larger than 1x images, so bandwidth-constrained UAs can opt  
to always download 1x images.


If it turns out that this is very useful, and that 1x images are still too  
large, then we may add more features for bandwidth negotiation.


Or perhaps bandwidth will increase fast enough that it won't be a  
significant problem by the time implementations reach users, or maybe  
high-dpi screens will become popular enough that 1x will become de-facto a  
low-bandwidth version, etc.


There’s a case to be made that few people implementing any form of  
“responsive images” solution will find a need to rely on an image’s  
intrinsic width, as doing so effectively negates any flexibility to  
begin with, which is almost entirely the point of either solution.


I don't understand how reliance on intrinsic size negates flexibility.

There’s no prior precedent this sort of thing—there’s no reason we can’t  
find a way to preserve an image’s intrinsic width using `picture`. I  
wonder if simply adding `width` and `height` attributes on the element  
(similar to `img`) might solve this, in the event that the author wants  
to rely on an intrinsic size instead of CSS?


I think that is a very good idea. Having option to do so is good for  
performance, as it avoids reflows.


--
regards, Kornel Lesiński

Re: [whatwg] Features for responsive Web design

2012-05-19 Thread Kornel Lesi��ski

On 19 maj 2012, at 10:46, Matthew Wilcox m...@matthewwilcox.com wrote:

 On 19 May 2012 00:37, Kornel Lesi��ski kor...@geekhood.net wrote:
 On Fri, 18 May 2012 23:11:45 +0100, Matthew Wilcox m...@matthewwilcox.com
 wrote:
 
 picture in its current form is unable to support bandwidth-based
 negotiation well
 
 
 By all accounts no solution proposed can do this. This is not a
 picture only problem.
 
 srcset allows UA to pick any image density regardless of actual screen
 density, so e.g. Safari on iPhone 4 on GPRS/EDGE connection could download
 1x images, instead of 2x.
 
 Yes indeed, but the problem is not about srcset, picture or anything
 else: it's that, as has been pointed out repeatedly to those of us
 asking about bandwidth media queries, the browser simply can't do
 reliable bandwidth detection. That effects srcset as much as anything
 else. So while technically srcset allows for this, the browser itself
 does't, so it can never be used in the manner you're describing.

iPhone knows when it's on GPRS/EDGE, so it can implement logic exactly as I've 
described it. 

What it doesn't know is exact numerical value representing current bandwidth 
and latency. 


 
 UAs are also free to download 1x image first, and then 2x image after
 onload, etc. Declarative nature of descriptors, as opposed to imperative
 MQs, allows UAs to innovate in this area and come up with new uses that
 authors didn't predict/express in MQ.
 
 This would be no better than current JS approaches, in fact making it
 worse by adding to the page load size rather than optimising it.

The point is UA can do whatever gives better experience. UAs can innovate in 
this area. For example phones know when they're on expensive roaming connection 
and always download lowest-res images, without needing authors to add 
(bandwidth-cost:expensive) MQ. 

 The syntax allows addition of KB descriptor later, which ― assuming UAs
 will figure out how to measure actual bandwidth well enough ― will allow
 even more sophisticated selection based on file size (rather than only 1x/2x
 scale factors which are only a proxy for the file size).
 
 As above; this seems set not to happen. Besides, how is it going to
 know the file size unless you explicitly state it in the srcset
 somewhere?

Setting explicit filesize may be an option. With current spec UA can safely 
assume that 1x image is smaller than 2x. 

 (and that's not simply a matter of adding bandwidth media
 query) and has no mechanism to specify scaling factor for intrinsic sizes
 of images.
 
 
 Not actually sure what intrinsic sizes of images means.
 
 
 A default size of the image when you don't specify any size in HTML/CSS.
 
 Ah, OK thanks :) I do not ever set intrinsic sizes, and nor does any
 site I have seen that is responsive in nature. Precisely because it
 has no meaning or use in a responsive site.

intrinsic size is the one you don't set. Size in img width or CSS width: 
overrides intrinsic size. 

 To take advantage of 200dpi displays you need to tell UA to render a X px
 image at X/2 CSS px. Explicit width/height breaks adaptive mechanism.
 
 Or you do what we do now and supply a double resolution image and set
 it to a given CSS size.

That requires relatively large amount of effort ― putting image sizes for all 
breakpoints in CSS, with same media queries, and ensuring you have unique 
selector for the image.

That's 5-10 lines of code for what you get with mere 2 characters in srcset. 

And when intrinsic size is usable you can use it for images with variable size 
(e.g. known width, but height dependent on aspect ratio of the image)

 Or just any old size, from my experience of
 the iPad3 you don't actually need to be that specific, just get an
 oversized image and let it shrink a bit and it holds up well.

That is true for devices which use zoom a lot, but may be suboptimal on 
desktops. 

 
 I see no difference between the end result of either syntax. Both
 approaches fail utterly in abstracting the query from the mark-up,
 which means both approaches are suited only to individual images.
 
 True. An URI template can be added later to either solution.
 
 I'd like to see idea's on how this might work.

Just throwing an idea here:

img srcset=gravatar?s={grsize}

style
@breakpoint grsize:80;

@media (max-width:320px) {
   @breakpoint grsize:40;
}
/style

I'm not sure about using style, as it cannot work in external CSS (we can't 
ask UAs to wait).
However declaration may need more syntax than just meta to allow separate 
groups of breakpoints. 

 
 And meta breakpoint has same limitations for DPI adaptation as source
 media.
 
 There are two categories of use-cases (art-directed and
 dpi/optimization) and picture is suited for only one of them, so
 please don't frame the decision as discarding of a perfectly good
 solution, as it wasn't.
 
 
 Picture dealt with both of these by simple use of the pixel density
 media query - i.e, in exactly the same way CSS already does it. I do

Re: [whatwg] Bandwidth media queries

2012-05-18 Thread Kornel Lesi��ski

I think we may be talking past each other, as I don't see how your answers 
address the problems I'm trying to highlight. 

It's not enough to say it's a hard problem. It's not going to solve itself.

If you say media queries can be useful for bandwidth/quality use-cases, you 
need to actually specify how can they work.

I'm trying to show here that MQ model is very problematic, and won't work well 
*even if UA has perfectly accurate bandwidth information at all times*!

MQs are stateless and expected to match the same way globally, and that clashes 
with stateful and non-uniform nature of caches that should be taken into 
account. 

So please specifically address cases I've listed in my previous email. 


-- 
regards, Kornel


On 18 maj 2012, at 10:52, Matthew Wilcox m...@matthewwilcox.com wrote:

 Thanks for all the feedback everyone.
 
 I don't think it's going to stop people trying to do this though;
 there are already write-ups for doing bandwidth detection in JS to
 manipulate img assets:
 http://www.csskarma.com/blog/detecting-for-bandwidth/
 
 Tricky problem.
 I'm not sure if that was intended to be an answer to my message:
 http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-May/036005.html
 
 I don't think that's at decision stage, because nobody has defined what
 options are there to decide.
 
 The options I see are not compelling:
 
 1. let it match actual bandwidth and apply rules according to standard media
 query logic. This will suck, as the page design will flip and reload
 whenever wind blows, and cache will be wasted.
 
 2. let it match some average or sticky value of bandwidth according to
 standard MQ logic. This will suck less, but still it won't make optimal use
 of cache OR bandwidth, and page may get stuck in suboptimal bandwidth (e.g.
 you catch WiFi only momentarily and get 3G browser stuck with peak value or
 vice versa).
 
 3. Violate MQ logic and allow mixed queries on the page (e.g. if browser has
 cached image while it had high bandwidth, use the image, even if bandwidth
 has dropped since). That will allow UAs to use best quality images it can
 and eliminate redundant requests, but will create unpredictable,
 inconsistent nightmare for designers.
 
 
 That's why I think there needs to be alternative solution parallel to MQs.
 It's a shame that Respimg mailinglist is dead:
 
 http://lists.w3.org/Archives/Public/public-respimg/2012May/0003.html
 
 --
 regards, Kornel Lesi��ski

Re: [whatwg] Features for responsive Web design

2012-05-18 Thread Kornel Lesiński

On Fri, 18 May 2012 20:24:00 +0100, André Luís andreluis...@gmail.com  
wrote:



Make no mistake; this is not a pride or attachment thing, this is a
knowing the reasons thing. I personally don't think picture answers
things well enough, nor do I think srcset does. Not for general use
cases - but for specific one-off use cases, each has benefits.


Absolutely. And from what I read (and I admit not reading the entire
archive of messages, but still, prefer to say my piece anyway) the
main concern about picture was the potencial verbosity. Instead of
trying to solve this issue, it got discarded.


picture in its current form is unable to support bandwidth-based  
negotiation well (and that's not simply a matter of adding bandwidth media  
query) and has no mechanism to specify scaling factor for intrinsic sizes  
of images.


IMHO those are pretty serious drawbacks that limit its functionality,  
which is much worse than ugly, but gets job done state of srcset.


There are two categories of use-cases (art-directed and  
dpi/optimization) and picture is suited for only one of them, so  
please don't frame the decision as discarding of a perfectly good  
solution, as it wasn't.




srcset addresses both dpi/optimisation and art-directed use-cases. It has  
its own problems, such as confusing microsyntax, so suggestions how to  
improve this are welcome.


Lamenting picture and accusing WHATWG of wrongdoing is not productive.

If you'd like to see picture proposal succeed, then please help fixing  
its drawbacks. Make selection and embedding of 2x images easier. Give UA  
freedom to use cached higher-quality images when it can. Give UA freedom  
to choose images to minimize bandwidth or maximize quality. Reduce  
verbosity of most common cases.


I've tried to raise those issues on the Responsive Images group's  
mailinglist, but got no constructive feedback.


--
regards, Kornel Lesiński

Re: [whatwg] Features for responsive Web design

2012-05-18 Thread Kornel Lesiński

On Fri, 18 May 2012 23:11:45 +0100, Matthew Wilcox  
m...@matthewwilcox.com wrote:



picture in its current form is unable to support bandwidth-based
negotiation well


By all accounts no solution proposed can do this. This is not a
picture only problem.


srcset allows UA to pick any image density regardless of actual screen  
density, so e.g. Safari on iPhone 4 on GPRS/EDGE connection could download  
1x images, instead of 2x.


UAs are also free to download 1x image first, and then 2x image after  
onload, etc. Declarative nature of descriptors, as opposed to imperative  
MQs, allows UAs to innovate in this area and come up with new uses that  
authors didn't predict/express in MQ.


The syntax allows addition of KB descriptor later, which — assuming UAs  
will figure out how to measure actual bandwidth well enough — will allow  
even more sophisticated selection based on file size (rather than only  
1x/2x scale factors which are only a proxy for the file size).



(and that's not simply a matter of adding bandwidth media
query) and has no mechanism to specify scaling factor for intrinsic  
sizes of images.


Not actually sure what intrinsic sizes of images means.


A default size of the image when you don't specify any size in HTML/CSS.

To take advantage of 200dpi displays you need to tell UA to render a X px  
image at X/2 CSS px. Explicit width/height breaks adaptive mechanism.



I see no difference between the end result of either syntax. Both
approaches fail utterly in abstracting the query from the mark-up,
which means both approaches are suited only to individual images.


True. An URI template can be added later to either solution.

The current URI template proposal is limited. I've pointed out few cases  
for which a single global set of breakpoints is not sufficient. I'd be  
nice if you tried to extend the proposal to support those cases as well  
(e.g. {case} → {case:category} and define breakpoints per category such as  
'sidebar', 'gravatar', etc.)


And meta breakpoint has same limitations for DPI adaptation as source  
media.



There are two categories of use-cases (art-directed and
dpi/optimization) and picture is suited for only one of them, so  
please don't frame the decision as discarding of a perfectly good  
solution, as it wasn't.


Picture dealt with both of these by simple use of the pixel density
media query - i.e, in exactly the same way CSS already does it. I do
not understand your argument.


picture
source src=200px media=(min-device-pixel-ratio:2)
source src=100px
/picture

This will only choose between large pixelated image and small pixelated  
image, and will not make use of high display density.


I've explained it in more detail previously (using srcset as an example,  
but it all applies to min-device-pixel-ratio: MQ as well)

http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2012-May/036051.html

--
regards, Kornel Lesiński

Re: [whatwg] Features for responsive Web design

On Thu, 17 May 2012 02:29:11 +0100, Jacob Mather jmat...@itsmajax.com  
wrote:



As I said, I understand that it is a hard problem, but the question
is, is it the correct problem.

There are plenty of reasons not to do it, but is there any actual
reason that the approach is less correct than the current proposals?


Yes, trading off latency to save bandwidth is definitely an incorrect  
approach. Bandwidth will keep increasing much faster than latency  
decreases (and there are hard physical limits to decreasing latency, while  
bandwidth could go up to infinity).


On high-latency high-bandwidth connections (satellite, 3G/4G) it may  
already be cheaper to download all versions of all images than to wait for  
CSS to be able to select the right ones to load. Solution that requires  
page layout for image loading is a step backwards for performance.


--
regards, Kornel Lesiński

Re: [whatwg] So if media-queries aren't for determining the media to be used what are they for?

On Wed, 16 May 2012 21:11:41 +0100, Matthew Wilcox  
m...@matthewwilcox.com wrote:


What solution do you have in mind that would let you add a 'tv'  
breakpoint site-wide for all images that have been prepared for it,  
without need to
update code that embeds those images? And is that really saving much  
effort?

Wouldn't you have to revisit every page anyway to test the new layout?


That following link does it:


[1]http://www.w3.org/community/respimg/2012/05/13/an-alternative-proposition-to-and-srcset-with-wider-scope/


Sorry, you're right. I glanced over URI template and assumed it was for  
media=(case: breakpoint1) in source.


--
regards, Kornel Lesiński

Re: [whatwg] Bandwidth media queries

On Wed, 16 May 2012 19:48:04 +0100, Matthew Wilcox  
m...@matthewwilcox.com wrote:



First off I know that a number of people say this is not possible. I
am not wanting to argue this because I don't have the knowledge to
argue it - but I do want to understand why, and currently I do not.
Please also remember that I can only see this from an authors
perspective as I'm ignorant of the mechanics of how these things work
internally.

The idea is to have something like:

link media=min-bandwidth:0.5mps ... /
link media=min-bandwidth:1mps ... /
link media=min-bandwidth:8mps ... /

This make an obvious kind of sense to an author.


What would happen in this scenario:

I'm in a cafe with free WiFi browsing on my phone, I have 8mbps bandwidth.  
I open the page and read it (it could be a webapp or a long article that  
stays open for a long time).


I leave the cafe, my bandwidth drops to 1mbps, but I still have the same  
page open.


Should the design change? Should the browser throw away all high-res  
images it has downloaded and re-download them in poor quality?


What when I reload or browse to a next subpage? My browser still has  
high-res images in the cache. Should it now apply low-end design and  
re-download all images in poor quality?


--
regards, Kornel

[whatwg] Defaulting new image solution to 192dpi



My suggestion is that the srcset (or picture) should assume that images  
are 2x scale by default.



My reasoning behind is:

- we have img for easy embedding of 1x images today, but we don't have  
2x img for the future. Having to specify width/height in img all the  
time is annoying.


- highdpi displays will become dominant at some point, it's only a matter  
of time (they pretty much are already in high-end smartphones, and are  
going to appear in laptops next). Bandwidth is also going to be less of a  
concern, so it'll be rational and desirable to serve images for the 2x  
resolution only (and just rely on 96dpi displays scaling them down).


Necessity to specify 2x scaling all the time will become a bad default and  
a historical quirk (like the DOCTYPE), and a source of annoyance where  
accidentally omitted 2x syntax makes images large and pixelated.



So to future-proof the solution I think:

img src=1x.jpg srcset=2x.jpg

should be equivalent to:

img src=1x.jpg srcset=2x.jpg 2x


and I expect that it'll eventually be commonly used this way:

img srcset=2x.jpg

when people stop caring about bandwidth overhead for low-end displays.


The srcset still allows 1x scale to be specified, so this change doesn't  
take any functionality away.


--
regards, Kornel Lesiński

Re: [whatwg] So if media-queries aren't for determining the media to be used what are they for?

On Thu, 17 May 2012 14:16:24 +0100, Matthew Wilcox  
m...@matthewwilcox.com wrote:



That particular solution is, to my mind, the most flexible and useful
implementation I've seen, because it's really about breakpoint
management and abstraction - which is what all responsive elements
need in order to work together well and be future-friendly.

It does, no doubt, have some technical considerations and implications
I'm not aware of. I would love to see this worked on more, I don't
think there's much more I can add to it from my authors perspective,
it needs work from an implementor perspective. But as a pattern, it
has a lot of plus points going for it.


I'm trying to figure out how it's going to work in other situations than  
the happy case.



What if you want to provide copypasteable code snippet with an adaptive  
image? (like W3C Validator badges)


What if you're creating WYSIWYG editor for CMSes and want to have button  
for inserting adaptive images, but have no access to head?


What if you want to insert a single image with custom/unique breakpoints  
(say an infographic prepared by an agency which used different  
breakpoints) on a website that already has its own breakpoints defined?


I see no nice solution for case when authors put meta breakpoint in  
body — it'd either have re-evaluate and potentially reload images  
(wasted requests) or ignore the meta (annoying gotcha when HTML5 parser  
unexpectedly closes head or when people want to work around lack of  
access to the head)


How do you work with URLs you have no control over? e.g. you'd have to  
name your breakpoints 40 and 80 to have adaptive size of gravatar.com  
URLs.


What do you do when you have only two breakpoints for sidebar, but more  
for the main content? (mostly fixed-width sidebar with fluid main column)  
or if your page header adapts to portrait orientation, but images in main  
content only adapt to width?


--
regards, Kornel Lesiński

Re: [whatwg] Defaulting new image solution to 192dpi

On Thu, 17 May 2012 14:42:19 +0100, Andy Davies dajdav...@gmail.com  
wrote:



Try browsing the web on the new iPad today. That's how every display is
going to look like in 10, maybe 20 years. Then DPI negotiation will not  
be an option, it'll be absolute requirement for *every* *single* image.


HTML5 is designed for the next 50-100 years.


The last line is exactly why baking in an assumption on the defaults
isn't the right way to go.


I know, somebody will quote me on 192dpi is enough for everybody, but we  
have 1x scaling assumption baked in already, so I'm suggesting changing  
existing bad assumption to a less bad assumption.



I don't disagree that higher DPI resolutions will be come the norm but
then what are we going to do about lower DPI devices, serve them a
higher DPI than needed image and let them work it out rather than
serve them appropriate images?


Yes. You do the same for 256-color and monochrome displays today, because  
potential bandwidth savings are small and popularity/quality of those  
screens is not significant enough to care.



The Zombie Apocalypse is coming there will be plenty of lower DPI
screens around for a long time...


Bandwidth will increase to the point that serving highdpi image to  
everybody won't be an issue.


Would you care about 100KB image vs 200KB image on 1Gbit connection?  
Google Fiber project is experimenting with such speeds already.


The answer is no: it takes only 0.8ms more, so even with *100* such images  
on the page the delay is literally comparable to blink of an eye  
(http://www.wolframalpha.com/input/?i=100KB+%2F+1Gbit+*+100)


The scaling factor won't increase forever, it's only going to be  
increasing until it matches capabilities of human vision. Depending on  
screen and viewing distance 2x is already claimed to be perfect (Retina  
display).


--
regards, Kornel Lesiński

Re: [whatwg] So if media-queries aren't for determining the media to be used what are they for?

On Thu, 17 May 2012 15:13:58 +0100, Matthew Wilcox  
m...@matthewwilcox.com wrote:



I'd also point out I'm not claiming this is a replacement for srcset
or picture. I think it's a useful additional tool for web
developers, but it is aimed at *site wide* generalised uses to make
life a lot easier - I can't see a way to make it also cater to
specific individual instances without fundamentally breaking the
benefits of the generalisation.


I see. So we could proceed with the spec without it, and add it later.

What if you're creating WYSIWYG editor for CMSes and want to have  
button for inserting adaptive images, but have no access to head?


This is not a valid use case (to my mind). You'd not be choosing
response points for individual pictures, it is the design that has
response points and it's the CSS (and JS/HTML) which fit into the
design's breakpoints.


OK.

I see no nice solution for case when authors put meta breakpoint in  
body

— it'd either have re-evaluate and potentially reload images (wasted
requests) or ignore the meta (annoying gotcha when HTML5 parser  
unexpectedly

closes head or when people want to work around lack of access to the
head)


This solution does not allow for meta in the body, and nor does the
HTML5 spec unless I'm mistaken?


What spec allows and what happens on the web are separate things :)

HTML needs to define all error cases and be prepared for markup to be  
abused (lots of such cases had to be defined already).


How do you work with URLs you have no control over? e.g. you'd have to  
name
your breakpoints 40 and 80 to have adaptive size of gravatar.com  
URLs.


I'm not sure I understand this point. However you cook it, you have no
access to those images so you can't do anything with them regardless
of the method of adaption, be it this method, srcset, or picture.


Yes, you can. gravatar.com allows you to specify desired size:

gravarar.com/avatar/hash?s=40 # gives 40x40px image
gravarar.com/avatar/hash?s=80 # gives 80x80px image


What do you do when you have only two breakpoints for sidebar, but more  
for the main content? (mostly fixed-width sidebar with fluid main  
column) or if your page header adapts to portrait orientation, but  
images in main content only adapt to width?


Breakpoints are not something that apply to individual components,
they apply to the page as a whole?


In my designs I use it mainly for components. I don't think of pages as  
separate phone/tablet/desktop designs, but as fluid designs which fold and  
unfold component by component.



Let's say my fancy header logo is quite tall, and that doesn't look well  
in landscape orientation, so I want to use short/wide version of the  
design in landscape orientation.


And I have a square illustration in my content. It doesn't flip on  
orientation, but I want it to be 500x500px on 1000px+ screens, and  
320x320px on smaller ones.


In some srcset-source pseudocode I'd say:

header
img src=logo-tall srcset=logo-short (orientation:landscape)
/header
acticle
	img src=illustration-big srcset=illustration-small  
(max-width:1000px)

/article

How would you do that with breakpoints?

--
regards, Kornel Lesiński

Re: [whatwg] Defaulting new image solution to 192dpi

On Thu, 17 May 2012 16:05:13 +0100, Tab Atkins Jr. jackalm...@gmail.com
wrote:

Necessity to specify 2x scaling all the time will become a bad default
and a historical quirk (like the DOCTYPE), and a source of annoyance
where

accidentally omitted 2x syntax makes images large and pixelated.

I think that 2x only looks like a good default now. I would bet that
in less than 10 years 3x or higher will look like a good default.

So why is 1x the default when it's obvious that it is not going to be a
good default?

If density is still absent, set it to 1.0.
http://www.whatwg.org/specs/web-apps/current-work/multipage/embedded-content-1.html#image-candidate-string

Note that the scale multiplier can be omitted already when only the size
is specified:

img srcset=img 500w

This is currently a legal syntax for a 1x image, and this is going to be
an awful mistake regardless whether we'll have 2x or 3x displays in the
future.

I'd rather not bake in a confusing change that doesn't actually
future-proof anything.

I think it's a safe bet that 3x won't ever be a good default.

Authors will need to translate between CSS and image pixel sizes, and it's
much easier to multiply and divide by 2 than 3.

Keep in mind that scaling factor is not tied too much to actual display
DPI/perceived pixel size. When new denser displays come along, we won't
jump from 2x to 3x, rather the 2x will change its meaning.

This is exactly what happened with pixel size at 1x — it has shrunk over
the years from 72dpi (SVGA 14) to 108dpi (iMac 27) and 130dpi on
laptops. Assuming same viewing distances, that's a hefty 1.5-1.8x increase
in resolution!

When resolution of current desktops doubles we'll start at around 200dpi,
and the pixel size will likely continue to shrink gradually over time as
displays get less-than-full-multiplier increases in density.

If such 1.5-1.8 increase happens again, the meaning of 2x will end up
being in the 300-360dpi range for desktop viewing distances. That will be
much better than 264dpi on the Retina iPad today (even better
considering iPad's smaller viewing distance).

Those pixels will be much smaller than average human can see — I think
that's perfectly adequate for a default!

Further improvements over this would have diminishing results — once
pixels are indistinguishable it won't make sense to make them more
indistinguishable, so there will be very little reason to use 3x, 4x
factors.

But let's say I'm wrong and 3x will end up being the ultimate scaling
factor:

- when authors specify the 3x scaling factor explicitly, it'll all work
fine regardless of the default.

- if the we don't change the default, then failing to specify the 3x scale
factor will give very wrong result — a 1x image.

- if we change the default to 2x, then failing to specify the 3x scale
factor will give less wrong result — a 2x image.

Only if we change the default to 3x now, before srcset ships we will avoid
the problem entirely. But as I've explained above, it's an inconvenient
scaling factor and is likely to overshoot required quality, so 2x is a
safer bet.

But why oh why should we keep 1x as the default if it's going to be the
worst default in any case?

--
regards, Kornel Lesiński

Re: [whatwg] Problems with width/height descriptors in srcset

On Thu, 17 May 2012 19:15:36 +0100, Matthew Wilcox  
m...@matthewwilcox.com wrote:



I also agree with Tab and Jeremy on this one - that makes a lot more
sense to me and removes any ambiguity without being overly verbose.


I like the less-compressed, more CSSy syntax as well.

I have two concerns though:

Is any media query allowed?
If so, then use of min-device-pixel-ratio:2x would give inferior results  
to using the 2x multiplier, because 2x allows UA to choose when the 2x  
image is selected (taking into account bandwidth, zoom, user preference,  
etc.), but min-device-pixel-ratio is supposed to query a fact about the  
device, not a preference. Should that be discouraged, or maybe even  
special-cased as an alias of 2x descriptor?



How do scaling factors and size queries interact?

When only scaling factors are specified then selection is easy, and images  
could be described in any order:


img srcset=imgA 1x, imgB 2x
img srcset=imgB 2x, imgA 1x


If first one that matches rule was applied to density descriptors, then  
this would surprisingly fail:

img srcset=imgA 1x, imgB 2x

because a 2x screen can display 1x image (or the opposite case would fail  
depending how you define matching of density).



As soon as sizes are specified, taking into account order of the rules  
becomes unavoidable:


img srcset=imgA min-height:100px, imgB min-width:100px

on a screen that is larger than 100x100 both would match, so there needs  
to be some definition which one wins, e.g. the first one that matches.


img srcset=imgA 1x min-height:100px, imgB 2x min-width:100px

but can UA choose the second image in this case when it prefers 2x density?


i.e. is the rule going to be first which matches at any density or  
first which matches at desired density, and failing that the first which  
matches at any other density?


--
regards, Kornel Lesiński

Re: [whatwg] Defaulting new image solution to 192dpi

On Thu, 17 May 2012 19:33:07 +0100, Tab Atkins Jr. jackalm...@gmail.com  
wrote:



I think that 2x only looks like a good default now. I would bet that
in less than 10 years 3x or higher will look like a good default.


So why is 1x the default when it's obvious that it is not going to be a  
good

default?


Because it's simple (1 image pixel = 1 CSS pixel), and it was the
historical default.


But it's going to become completely useless and nobody is going to use it.  
2x default at least has a chance of not being useless.


--
regards, Kornel Lesiński

Re: [whatwg] Defaulting new image solution to 192dpi

On Thu, 17 May 2012 19:32:51 +0100, Jeremy Keith jer...@adactio.com  
wrote:



Kornel wrote:
Note that the scale multiplier can be omitted already when only the  
size is specified


I'm confused by what you mean by scale multiplier. The x value describes  
the pixel density of the device/screen, not the image, right?


I think it only makes sense if it describes the image, so that:

img srcset=100x100image-pixels.jpg 2x

would display at 50x50px (CSS px).

Of course the UA is likely to match 2x with double-density displays, so it  
acts as both description of the image and a hint media query.



If it were purely a media query unrelated to the image, then this:

img srcset=100x100image-pixels.jpg 2x, 50x50image-pixels.jpg 1x

would only select between pixelated large image and pixelated small image,  
never making use of a high DPI display.



The goal is to avoid having to specify explicit img width/height. Not  
only it's tedious and annoying, it also makes it impossible to adapt  
images to viewport and screen density at the same time:


img srcset=
	phone-retina-640imgpx.jpg 2x max-width:320px, phone-regular-320imgpx.jpg  
1x max-width:320px,

desktop-1000imgpx.jpg 2x, desktop-500imgpx.jpg 1x


In the case above I'd want to have phone* images on =320px viewport,  
matching display density, and desktop* images on larger viewports, taking  
full advantage of display density as well (i.e. desktop-1000imgpx.jpg 2x  
displayed at 500px).


There is no value of img width= height= that can work for that case  
without ruining the adaptation img width=500 would work for desktop, but  
ruin phone versions.


So the only way to use srcset in that case would require additional media  
queries that react to same breakpoints as srcset and set explicit pixel  
sizes of this image for each breakpoint. Awful.


So for me the only logical conclusion is that images with 2x descriptor  
must have their intrinsic size halved (2 img pixels to 1 css pixel).  
That's what iOS does when you use @2x images in Cocoa.


--
regards, Kornel Lesiński

Re: [whatwg] So if media-queries aren't for determining the media to be used what are they for?

2012-05-16 Thread Kornel Lesiński

On Wed, 16 May 2012 20:09:13 +0100, D. Pitchford dpitchfo...@gmail.com
wrote:

What standards does not do in this situation is remove the actual work
effort in having to physically update each and every img's 'srcset'
string with new breakpoints during a redesign, no matter how terse the
'srcset' string is.
You can have all the stanadrds in place you can muster to write, but the
physical work effort is not negated because of a list of guidelines that
lives in a document that rarely gets followed.

Another situation could involve adding an additional layer of support
for Televisions, in the middle of the lifecycle of a website.
srcset does not allow for a global update to all img's in a templated
fashion. The work effort would need be physically updating each and
every srcset across the site individually.

Not exactly the most efficient use of anyone's time. (picture also has
this as a downfall)

This is a much larger issue than people are willing to admit at this
point.

The solution I've seen proposed[1] only aliases media query content, and
works only on a per-page basis, so it doesn't allow automatic addition of
a new image size site-wide, since you have to insert new source into
every picture anyway.

To me it looks like about the same amount of work as inserting new pixel
size into every srcset.

What solution do you have in mind that would let you add a 'tv' breakpoint
site-wide for all images that have been prepared for it, without need to
update code that embeds those images? And is that really saving much
effort? Wouldn't you have to revisit every page anyway to test the new
layout?

[1]http://www.w3.org/community/respimg/2012/05/13/an-alternative-proposition-to-and-srcset-with-wider-scope/

--
regards, Kornel Lesiński

Re: [whatwg] Features for responsive Web design

2012-05-16 Thread Kornel Lesiński

On Wed, 16 May 2012 20:12:19 +0100, Jacob Mather jmat...@itsmajax.com  
wrote:



Maybe this is the better question:

Why does the pre-loader matter so much?

Basing the selected image off of browser width is inherently
backwards. The content should be informed by the layout, not by the
browser.


Browsers want to download images in parallel with a stylesheet, i.e. they  
must know which images to load *before* CSS is loaded.


This is very important to browser vendors. Google goes even as far as  
replacing the HTTP protocol to remove those kinds of delays.



In addition to that applying layout to the page may be computationally  
expensive, so it would further delay time it takes to start loading images.


And finally corner cases can lead to loops:

div style=float:left
picture
source media=max-container-width:100px src=200px-wide-image.png
source media=min-container-width:100px src=50px-wide-image.png
/picture
/div

--
regards, Kornel Lesiński

Re: [whatwg] So if media-queries aren't for determining the media to be used what are they for?

2012-05-15 Thread Kornel Lesiński

On Tue, 15 May 2012 23:17:54 +0100, Chris Heilmann code...@gmail.com  
wrote:


The fetish for brevity is something I never understood. More  
understandable code is faster to write than cryptic short code.


There is significant difference in verbosity for a *very common case* of  
serving images for high-dpi (Retina) display (which I suspect is only  
going to get more common):


img src=lowdpi srcset=hidpi 2x

vs

picture
source media=(min-device-pixel-ratio: 2) src=hidpi
img src=lowdpi
/picture


It will get tiring when it'll have to be used for every image on the page.

Authors couldn't be bothered to type extra markup for all vendor's  
prefixes in CSS. Nobody bothered with verbose SVG gradient syntax which  
was usable before CSS gradients. HTML5 DOCTYPE is loved. Brevity matters.


--
regards, Kornel Lesiński

Re: [whatwg] So if media-queries aren't for determining the media to be used what are they for?

2012-05-15 Thread Kornel Lesiński

On Tue, 15 May 2012 23:57:48 +0100, Silvia Pfeiffer  
silviapfeiff...@gmail.com wrote:



Media queries come from the client side. They allow the author of a web
page to tell exactly how she want to lay out her design based on the
different queries. The browser *HAS* to follow these queries. And also,
I don't think (please correct me if wrong) the media query can be subset
to only the stuff that's really meaningful to do at prefetch-time.

The srcset proposal, on the other hand, are purely HINTS to the browser
engine about the resources. They are only declarative hints that can be
leveraged in a secret sauce way (like Bruce said in another mail) to
always optimize image fetching and other features. If you make a new
kind of browser (like e.g. Opera mini) it can have its own heuristics
that make sense *for that single browser* without asking _anyone_.
Without relying on web authors doing the correct thing, or changing
anything or even announce to anyone what they are doing. It's opening up
for innovation, good algorithms and smart uses in the future.


That's the basic difference, totally different. :-)


If that's the case, would it make sense to get rid of the @media
attribute on source elements in video and replace it with @srcset?


I think something like that would be perfect.

In fact, I'd keep @media, because it serves some cases very well (I see  
dpi/bandwidth optimisation as a problem orthogonal to layout adaptation:  
http://geekhood.net/MediaQuery-vs-PerfQuery.png)



It may be enough to add another attribute that describes image properties,  
and can be used either alone or with conjunction with MQs:


picture
source scale=1 src=low
source scale=2 src=high
/picture

would be equivalent to:

img srcset=low 1x, high 2x

(I've proposed that in the RespImg community group)


And combination of the two opens new possibilities:

picture
source scale=1 src=low-narrow media=(orientation:portrait)
source scale=2 src=high-narrow media=(orientation:portrait)
source scale=1 src=low-wide media=(orientation:landscape)
source scale=2 src=high-wide  media=(orientation:landscape)
/picture

--
regards, Kornel Lesiński

Re: [whatwg] Features for responsive Web design

2012-05-15 Thread Kornel Lesiński

On Tue, 15 May 2012 19:25:23 +0100, Matthew Wilcox  
m...@matthewwilcox.com wrote:



I think there's a fundamental mis-match in the mental model of how
authors work and what they want. I'm pretty sure we're all shooting
for the same be more efficient goal, but I think that here on the
mailing list that's being approached from an angle that has not
considered how authors actually want to do this.

We work with designs that re-arrange content and sometimes call for
different images of the same semantic meaning. That is *not* the same
use case as simply sending a different version of the same image.
Srcset only addresses that one type of use, and that is why authors
feel it's flawed. It doesn't do what we need, and never can because
srcset is based on the assumptin that a UA can somehow pick an
appropriate resource to load - when it can't possibly know about the
authors use of that resource at that time.


There's very good article about the two cases:

http://blog.cloudfour.com/a-framework-for-discussing-responsive-images-solutions/

srcset is not very good for art-directed case, while picture is  
perfect for it.


picture is not very good for resolution/bandwidth optimisation, while  
srcset is perfect for it.



I think those are simply two different problems that just happen to be  
called adaptive images. We should recognized that they're separate and  
design separate solutions for them. A single solution can't do both well,  
since there's a fundamental difference between author-controlled and  
UA-controlled decision.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-14 Thread Kornel Lesiński

On Mon, 14 May 2012 01:30:20 +0100, Odin Hørthe Omdal odi...@opera.com  
wrote:



All optional replacements of the src will have to be fitted in the same
box as the original src. That might actually require you to specify both
width and height upfront. Of course, people won't really do that, so I
guess we're bound to get differing behaviour... Hm.

What do people think about that? What happens here? You have no info on
the real size of the picture. I guess maybe the browser should never
load any srcset alternatives then? If you have no information at all
it's rather hard to make a judgement.

A photo gallery wants to show you a fullscreen picture, and give you:

   img src=2048px.jpg srcset=4096px.jpg 2x

In this example, us (humans :P) can easily see that one is 2048 px and  
the other 4096 px. If I'm viewing this on my highres Nokia N9, a naïve

implementation could pick the 2x, because it knows that's nicely highres
just like its own screen.

But it would actually be wrong! It would never need anything else than
the 2048 px for normal viewing because it is anyway exceeding its real
pixels on the screen.


If srcset/picture provides authors with good way to serve images at most  
appropriate size, they won't need to resort to tricks with downsizing  
high-res images to smaller size.


For a full-width image on a ~960px viewport (assuming author doesn't have  
better sizes available) this would be appropriate:


  img src=2048px.jpg srcset=2048px.jpg 2x, 4096px.jpg 4x  
style=width:100%



--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 18:01:12 +0100, Benjamin Hawkes-Lewis  
bhawkesle...@googlemail.com wrote:



What authors _can_ do and user agents _cannot_ do is describe their
images. Such metadata never needs to be misinterpreted and allows user
agents to iterate and improve the end-user experience even when the
author either does not care about them, or has moved on, or is long
dead.


Indeed. An interesting use-case has come up on this list — use  
highest-resolution image when user chooses Save As.


While it's not an earth-shattering feature, it's a nice touch a browser  
could do having information about image. It wouldn't be possible/sensible  
to do with a media query.


Even if there was support for something like source  
media=user-action:save-as, I can bet that less than 0.5% of pages would  
use it and do it correctly.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 10:36:37 +0100, Jason Grigsby ja...@cloudfour.com  
wrote:



On May 13, 2012, at 9:51 AM, David Goss wrote:


A common sentiment here seems to be that the two proposed responsive
image solutions solve two different use cases:


After skyping with Mat (@wilto) last night, I think I may be the only  
one who didn’t fully grok that the mediaqueries in picture could be  
used to address both use cases.


It is still unclear to me if img srcset would address both.


I think layout (media queries) and optimisation cases are orthogonal and  
it would be a mistake to do both with the same mechanism.


Adaptation of images to the layout is page-specific. Adaptation of images  
to bandwidth/screen is UA/device-specific.


Author is in the best position to adapt image to page layout. User-agent  
is in the best position to determine speed/quality trade-offs.


Media queries MUST be interpreted exactly as author specified them.  
User-agents need freedom to choose image resolution based on open set of  
factors, many of which are details authors should not have to think about  
(presence in cache, cost of bandwidth, available memory, external  
displays, etc.)



So even though both proposals are about selecting right images, their  
purposes and requirements are very different.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 01:33:25 +0100, Mathew Marquis m...@matmarquis.com  
wrote:


I worry that, when faced with this markup, developers will simply opt to  
serve the largest possible image in a src. In fairness, that approach  
works with far less headache.


In the long term that may be a very sensible approach. Selection of 1x/2x  
images is relevant only as long as we have 100dpi screens and slow  
connections, and both will disappear over time.


Perhaps we should think about making syntax for 200dpi+ images with  
intrinsic dimensions easy?



Selection between images for different screen sizes/orientations isn't a  
problem that will go away any time soon, so I think picture is a good  
idea regardless of solution for DPI/bandwidth problem.
As long as HTML claims to be independent of CSS there is no solution for  
that (i.e authors shouldn't be adding multiple img and showing one of  
them with CSS).




How about that:

picture
	source src=narrow_low-quality srcset=narrow_hi-quality 2x  
media=max-width:4in

source src=wide_low-quality srcset=wide_hi-quality 2x

img src=fallback alt=alt
/picture


Instead of srcset it could be src2x or another attribute that specifies  
image for higher screen density and/or bandwidth. The point is that  
media= would allow author to choose image version adapted to page  
layout, and another mechanism connected to source would allow UA to  
choose image resolution.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 20:20:54 +0100, Mathew Marquis m...@matmarquis.com  
wrote:



The key problem about designing a responsive images solution around
user agent characteristics not image characteristics is that authors
will inevitably make more false assumptions about what images match
what user agent characteristics than user agents will. As a result,
user agents may be forced to misinterpret media queries in order to
provide their users with better user experiences.


Correct me if I’m wrong, but wouldn’t this same reasoning apply to  
layouts that depend heavily on media queries?


No, those are very different situations.

Layout is under author's control and is known to the author when the page  
is authored. UA characteristics are not, and different UAs will have  
different requirements.


Conversely from UA perspective: layouts vary wildly between pages, so UA  
can't really mess with them. Bandwidth/DPI requirements are known to UA  
and there can be same set of UA-specific rules that works for all pages.


--
regards, Kornel Lesiński

[whatwg] Fallback for picture



Syntax used on the wiki:
http://wiki.whatwg.org/wiki/Adaptive_images

places alt on the new element:

picture alt=alt
source …
img
/picture


I think it can be improved in two ways:

- Instead of having alt on picture, it could be on the fallback img.  
This will give better backward-compatibility.


- Use of an attribute for alternative content is very limiting, e.g. image  
of a comic cannot have dialog marked up well. Use of an non-empty element  
opens up possibility of richer alternatives.



The processing rules for extracting fallback from picture would be:

1. Take all children of picture
2. Remove/ignore all source elements.
3. Interpret all img alt= elements as their alt text.


picture
source …
img alt=This is unstructured fallback
/picture

and

picture
source …
img
This is emstructured/em fallback
/picture

The two examples above would have This is unstructured fallback and  
This is emstructured/em fallback as their alt, respectively.



A use case for markup in alt:

picture
source  
src=world-map-showing-most-popular-browser-in-each-country.png

tabletrthCountry/ththMost popular browser/th...
/picture

Trying to put all data in alt= wouldn't work well, and

img alt=world map showing most popular browser in each country

doesn't contain the information that the map conveys, so that's at best a  
caption, not an alternative.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 20:55:08 +0100, Bjartur Thorlacius  
svartma...@gmail.com wrote:



The problem with that, though, is that then bandwidth constraints
can't affect layout. Users should be able to configure UAs to use
downsized images even given a large viewport, if only to save
bandwidth and reserve a larger fraction of the viewport for text
columns.


That wasn't my assumption.

For optimisation case expected images to vary only in compression strength  
or DPI (low-res JPEG or 2x image for Retina display), but never in their  
dimension in CSS pixels.



I think page should not be allowed to change layout based on bandwidth  
availability, because of browser caches. Imagine scenario:


1. User visits page on high-bandwidth connection and UA caches some high  
quality images.

2. Connection changes to slower one.
3. User visits the page again. HTML was non-cacheable, but images were  
cacheable.


Now the page may have mix of high-bandwidth and low-bandwidth content.  
It's entirely possible to have mixed quality with images if they have same  
CSS pixel size (and use of higher quality images whenever possible is  
desirable), but a page cannot contain mix of multiple different layouts at  
the same time.


If bandwidth was a media query, then entire page may have to be downgraded  
to low-bandwidth version even if UA had a lot of high-bandwidth content  
cached.


Adaptation of images to the layout is page-specific. Adaptation of  
images to bandwidth/screen is UA/device-specific.



Quite. But the latter just might affect the layout.


I think optimisation case should be forbidden from affecting the layout.


Author is in the best position to adapt image to page layout. User-agent
is in the best position to determine speed/quality trade-offs.


But low-res images usually don't look too good when upscaled. Thus few
pixels should mean small image, UAs mustn't default to pixelation.


I don't understand why UA would default to pixelation?

The whole point of declaring available image versions is giving UA  
possibility to default to whatever it deems best.


When user is on GPRS or roaming connection then pixelation is a great  
outcome compared to not being able to download images at all due to  
prohibitive size/price.


When user is on broadband connection then UA can select high resolution  
images and avoid pixelation when network/hardware allows it.


And when image properties are specified declaratively, a smart UA can even  
do a mix of both, e.g. download pixelated images on mobile connections and  
give user option to selectively download higher-res images, or download  
low-res first for fast initial display and then download high-res in the  
background.



Media queries MUST be interpreted exactly as author specified them.

Thus we mustn't force UAs to pretend to render to small viewports to
find low-res images. That would have unwieldy side-effects.


Indeed!


User-agents need freedom to choose image resolution based on open set of
factors, many of which are details authors should not have to think  
about (presence in cache, cost of bandwidth, available memory, external

displays, etc.)


But the chosen image resolution might be a factor for choosing layout.


By resolution I mean pixel density (regular vs Retina display), so this  
doesn't affect layout.



I can imagine layout complexity being tied to bandwidth (an image-rich  
design vs minimalistic text-only design), but I'm not sure how that would  
work in practice given that cache has infinite bandwidth, and network  
speed can change any second on mobile connections.


It would be weird if page design changed when you moved between cell  
towers or left/entered a cafe that had public WiFi. And if bandwidth media  
query was defined to be fixed, then you'd sometimes end up stuck with  
wrong design that was chosen based on a temporary network state.


There is no such problem if only same-CSS-pixel-size images are swapped  
in-place.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 21:23:58 +0100, Odin Hørthe Omdal odi...@opera.com  
wrote:



 picture
  source src=narrow_low-quality srcset=narrow_hi-quality 2x
 media=max-width:4in
  source src=wide_low-quality srcset=wide_hi-quality 2x

  img src=fallback alt=alt
 /picture

Instead of srcset it could be src2x or another attribute that specifies
image for higher screen density and/or bandwidth. The point is that
media= would allow author to choose image version adapted to page
layout, and another mechanism connected to source would allow UA to
choose image resolution.


Seeing it here in code it's actually not such a monster that I'd said
it'd be. So I like it even more, and it's the obvious way for these to
interact.

I think it'd be a mistake to call it src2x though, -- it feels very
specific. You can scale up to double then, but you can't necessarily go
beyond that: going down for e.g. mobile.

OTOH, 2x will be the most common usage at least as far as I can tell.

  img src=dog.jpg src2x=d...@2.jpg

  vs.

  img src=dog.jpg srcset=d...@2.jpg 2x

is not really all that different, but the second should be more
flexible. Also downscaling:

  img src=dog.jpg srcset=d...@2.jpg 2x, dog-lo.jpg 0.5x


Yes, good point.


Actually, for this to work, the user agent needs to know the size of the
standard image. So:

  img src=dog.jpg width=960
srcset=d...@2.jpg 2x, dog-lo.jpg 500w

So if you've got the smartphone held in portrait, it's 250 css pixels
wide, and so 500 real pixels, it could opt to show dog-lo.jpg rather
than dog.jpg.


But still displayed at 960 CSS pixels or course? That'd be fine (and the  
UA could even download dog@2x when user zooms in).


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)