Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"

2014-05-27 Thread Joshua Bell
On Fri, May 23, 2014 at 6:24 PM, marc fawzi  wrote:

> Here is a jsfiddle showing how .advance behaves when the range is
> restricted by .only
>
> Create new e.g. 7 items with names like "marc" and tags like "w1 w3 w5 w2"
> (random selection of tags with some tags appearing across multiple records
> (per the attached image)
>
> Enter "w2" or "w5" in the box next to 'get by tag' and click 'get by tag'
>
> You'll see the first 2 matching items, with primary keys 7 and 6
>
> Click 'get by tag' again and you'll see the next 2 matching items, with
> primary keys 4 and 2
>
> Click 'get by tag' again and you'll see the next and last matching item,
> with primary key 1
>
> Notice the way I advance the cursor each time in order to re-continue the
> search from where I left off in the previous invocation is by using the
> number of items already found
>
> http://jsfiddle.net/marcfawzi/y5ELj/
>
>
Thanks for sharing this example.


> It's the correct behavior but it would be easier imo if we have .find()
> for what .continue() and .continue(key) does and use .continue() to mean
> .advance(1) and .continue(n) to mean .advance(n)
>
> But I could be totally wrong. Just a harmless feedback at this point.
>
>
It's a reasonable suggestion. Unfortunately, we already have multiple
shipping implementations and code in the wild depending on the API as
specified, and this would be a breaking change. It's useful feedback if we
add new cursor/iteration APIs in the future, though - the current choice of
"continue" and "advance" vs. e.g. "find" or "seek" is pretty arbitrary and
can be a source of confusion.

Thanks again for following up with examples to ensure we understood your
feedback!


> :)
>
>
> On Fri, May 23, 2014 at 1:07 PM, marc fawzi  wrote:
>
>> <<
>> Thanks for following up! At least two IDB implementers were worried that
>> you'd found some browser bugs we couldn't reproduce.
>> >>
>> Yup. I had to figure this stuff out as the API is very low level (which
>> is why it can also be used in very powerful ways and also potentially very
>> confusing for the uninitiated)
>>
>> <> not restricted, if the cursor's key=7 and direction='prev' then I would
>> expect after advance(2) that key=5. If you're seeing key=2 can you post a
>> sample somewhere (e.g. jsfiddle.com?)>>
>>
>> In the case I have say 7 items [1,2,3,4,5,6,7] and the cursor's range is
>> restricted by IDBKeyRange.only(val, "prev") ... so if the matching (or in
>> range) items are at 7, 6, 4, 2, 1 then I can obtain them individually or in
>> contiguous ranges by advancing the cursor on each consecutive invocation of
>> my search routine, like so: on first invocation advance(1) from 7 to 6, on
>> second invocation advance(2) from 7 to 4, on third invocation advance(3)
>> from 7 to 2 and on fourth invocation advance(4) from 7 to 1. I could also
>> use advance to advance by 1 within each invocation until no matching items
>> are found but only up to 2 times an invocation (for a store with 700 or
>> 7 items we can advance by 1 about 200 times per invocation, but that's
>> arbitrary)
>>
>>  I can definitely post a jsfiddle if you believe the above is not in
>> accordance with the spec.
>>
>> As to continue(n) or continue(any string), i would make that
>> .find(something)
>>
>>
>>
>> On Fri, May 23, 2014 at 10:41 AM, Joshua Bell  wrote:
>>
>>> On Fri, May 23, 2014 at 9:40 AM, marc fawzi wrote:
>>>
 I thought .continue/advance was similar to the 'continue' statement in
 a for loop in that everything below the statement will be ignored and the
 loop would start again from the next index. So my console logging was
 giving confusing results. I figured it out and it works fine now.

>>>
>>> Thanks for following up! At least two IDB implementers were worried that
>>> you'd found some browser bugs we couldn't reproduce.
>>>
>>>
  For sanity's sake, I've resorted to adding a 'return'  in my code in
 the .success callback after every .advance and .continue so the execution
 flow is easier to follow. It's very confusing, from execution flow
 perspective, for execution to continue past .continue/.advance while at
 once looping asynchronously. I understand it's two different instances of
 the .success callback but it was entirely not clear to me from reading the
 docs on MDN (for example) that .advance / .continue are async.

>>>
>>> Long term, we expect JS to evolve better ways of expressing async calls
>>> and using async results. Promises are a first step, and hopefully the
>>> language also grows some syntax for them. IDB should jump on that train
>>> somehow.
>>>
>>>
 Also, the description of .advance in browser vendor's documentation,
 e.g. on MDN, says "Advance the cursor position forward by two places" for
 cursor.advance(2) but what they should really say is "advance the cursor
 position forward by two results." For example, let's say cursor first
 landed on an item with primary key = 7, and you is

Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"

2014-05-23 Thread marc fawzi
<<
Thanks for following up! At least two IDB implementers were worried that
you'd found some browser bugs we couldn't reproduce.
>>
Yup. I had to figure this stuff out as the API is very low level (which is
why it can also be used in very powerful ways and also potentially very
confusing for the uninitiated)

<>

In the case I have say 7 items [1,2,3,4,5,6,7] and the cursor's range is
restricted by IDBKeyRange.only(val, "prev") ... so if the matching (or in
range) items are at 7, 6, 4, 2, 1 then I can obtain them individually or in
contiguous ranges by advancing the cursor on each consecutive invocation of
my search routine, like so: on first invocation advance(1) from 7 to 6, on
second invocation advance(2) from 7 to 4, on third invocation advance(3)
from 7 to 2 and on fourth invocation advance(4) from 7 to 1. I could also
use advance to advance by 1 within each invocation until no matching items
are found but only up to 2 times an invocation (for a store with 700 or
7 items we can advance by 1 about 200 times per invocation, but that's
arbitrary)

 I can definitely post a jsfiddle if you believe the above is not in
accordance with the spec.

As to continue(n) or continue(any string), i would make that
.find(something)



On Fri, May 23, 2014 at 10:41 AM, Joshua Bell  wrote:

> On Fri, May 23, 2014 at 9:40 AM, marc fawzi  wrote:
>
>> I thought .continue/advance was similar to the 'continue' statement in a
>> for loop in that everything below the statement will be ignored and the
>> loop would start again from the next index. So my console logging was
>> giving confusing results. I figured it out and it works fine now.
>>
>
> Thanks for following up! At least two IDB implementers were worried that
> you'd found some browser bugs we couldn't reproduce.
>
>
>>  For sanity's sake, I've resorted to adding a 'return'  in my code in the
>> .success callback after every .advance and .continue so the execution flow
>> is easier to follow. It's very confusing, from execution flow perspective,
>> for execution to continue past .continue/.advance while at once looping
>> asynchronously. I understand it's two different instances of the .success
>> callback but it was entirely not clear to me from reading the docs on MDN
>> (for example) that .advance / .continue are async.
>>
>
> Long term, we expect JS to evolve better ways of expressing async calls
> and using async results. Promises are a first step, and hopefully the
> language also grows some syntax for them. IDB should jump on that train
> somehow.
>
>
>> Also, the description of .advance in browser vendor's documentation, e.g.
>> on MDN, says "Advance the cursor position forward by two places" for
>> cursor.advance(2) but what they should really say is "advance the cursor
>> position forward by two results." For example, let's say cursor first
>> landed on an item with primary key = 7, and you issue the statement
>> cursor.advance(2), I would expect it to go to the item with primary key 5
>> (for cursor direction = "prev") but instead it goes to the item with
>> primary key 2 because that's the 2nd match for the range argument from the
>> cursor's current position
>>
>
> What "range argument" are you referring to?
>
> Assuming the store has [1,2,3,4,5,6,7,8,9] and the cursor's range is not
> restricted, if the cursor's key=7 and direction='prev' then I would expect
> after advance(2) that key=5. If you're seeing key=2 can you post a sample
> somewhere (e.g. jsfiddle.com?)
>
>
>> , which means that .advance(n) would be far more clear semantically
>> speaking if it was simply done as .continue(n)  ... I guess if there is an
>> understanding that the cursor is always at a matching item and that it
>> could only continue/advance to the next/prev matching item, not literal
>> 'positions' in the table (i.e. sequentially through the list of all items)
>> then there would be no confusion but the very concept of a cursor is
>> foreign to most front end developers, and that's where the confusion comes
>> from for many.
>>
>> My inclination as a front end developer, so far removed from database
>> terminology, would be
>>
>> 1) to deprecate .advance in favor of .continue(n) and
>>
>
> continue(n) already has meaning - it jumps ahead to the key with value n
>
>
>>
>> 2) if it makes sense (you have to say why it may not) have
>> .continue()/.continue(n) cause the return of the execution flow similar to
>> 'continue' in a for loop.
>>
>
> The API can't change the language - you return from functions via return
> or throw. Further, there are reasons you may want to do further processing
> after calling continue() - e.g. there may be multiple cursors (e.g. in a
> join operation) or for better performance you can call continue() as early
> as possible so that the database can do its work while you're processing
> the previous result.
>
>
>
>>
>> What do you think?
>>
>>
>>
>> On Wed, May 21, 2014 at 10:42 AM, Joshua Bell  wrote:
>>
>>>
>>>
>>>
>>> On Wed, May 21, 2014 at 7:

Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"

2014-05-23 Thread Joshua Bell
On Fri, May 23, 2014 at 9:40 AM, marc fawzi  wrote:

> I thought .continue/advance was similar to the 'continue' statement in a
> for loop in that everything below the statement will be ignored and the
> loop would start again from the next index. So my console logging was
> giving confusing results. I figured it out and it works fine now.
>

Thanks for following up! At least two IDB implementers were worried that
you'd found some browser bugs we couldn't reproduce.


> For sanity's sake, I've resorted to adding a 'return'  in my code in the
> .success callback after every .advance and .continue so the execution flow
> is easier to follow. It's very confusing, from execution flow perspective,
> for execution to continue past .continue/.advance while at once looping
> asynchronously. I understand it's two different instances of the .success
> callback but it was entirely not clear to me from reading the docs on MDN
> (for example) that .advance / .continue are async.
>

Long term, we expect JS to evolve better ways of expressing async calls and
using async results. Promises are a first step, and hopefully the language
also grows some syntax for them. IDB should jump on that train somehow.


> Also, the description of .advance in browser vendor's documentation, e.g.
> on MDN, says "Advance the cursor position forward by two places" for
> cursor.advance(2) but what they should really say is "advance the cursor
> position forward by two results." For example, let's say cursor first
> landed on an item with primary key = 7, and you issue the statement
> cursor.advance(2), I would expect it to go to the item with primary key 5
> (for cursor direction = "prev") but instead it goes to the item with
> primary key 2 because that's the 2nd match for the range argument from the
> cursor's current position
>

What "range argument" are you referring to?

Assuming the store has [1,2,3,4,5,6,7,8,9] and the cursor's range is not
restricted, if the cursor's key=7 and direction='prev' then I would expect
after advance(2) that key=5. If you're seeing key=2 can you post a sample
somewhere (e.g. jsfiddle.com?)


> , which means that .advance(n) would be far more clear semantically
> speaking if it was simply done as .continue(n)  ... I guess if there is an
> understanding that the cursor is always at a matching item and that it
> could only continue/advance to the next/prev matching item, not literal
> 'positions' in the table (i.e. sequentially through the list of all items)
> then there would be no confusion but the very concept of a cursor is
> foreign to most front end developers, and that's where the confusion comes
> from for many.
>
> My inclination as a front end developer, so far removed from database
> terminology, would be
>
> 1) to deprecate .advance in favor of .continue(n) and
>

continue(n) already has meaning - it jumps ahead to the key with value n


>
> 2) if it makes sense (you have to say why it may not) have
> .continue()/.continue(n) cause the return of the execution flow similar to
> 'continue' in a for loop.
>

The API can't change the language - you return from functions via return or
throw. Further, there are reasons you may want to do further processing
after calling continue() - e.g. there may be multiple cursors (e.g. in a
join operation) or for better performance you can call continue() as early
as possible so that the database can do its work while you're processing
the previous result.



>
> What do you think?
>
>
>
> On Wed, May 21, 2014 at 10:42 AM, Joshua Bell  wrote:
>
>>
>>
>>
>> On Wed, May 21, 2014 at 7:32 AM, Arthur Barstow wrote:
>>
>>> [ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ]
>>>
>>> On 5/20/14 7:46 PM, marc fawzi wrote:
>>>
 Hi everyone,

 I've been using IndexedDB for a week or so and I've noticed that
 cursor.advance(n) will always move n items forward regardless of cursor
 direction. In other words, when the cursor direction is set to "prev" as
 in: range = IDBKeyRange.only(someValue, "prev") and primary key is
 auto-incremented, the cursor, upon cursor.advance(n), will actually advance
 n items in the opposite direction to the cursor.continue() operation.
>>>
>>>
>> That runs contrary to the spec. Both continue() and advance() reference
>> the "steps for iterating a cursor" which picks up the direction from the
>> cursor object; neither entry point alters the steps to affect the direction.
>>
>> When you say "you've noticed", are you observing a particular browser's
>> implementation or are you interpreting the spec? I did a quick test and
>> Chrome, Firefox, and IE all appear to behave as I expected when intermixing
>> continue() and advance() calls with direction 'prev' - the cursor always
>> moves in the same direction regardless of which call is used.
>>
>> Can you share sample code that demonstrates the problem, and indicate
>> which browser(s) you've tested?
>>
>>
>>
>>
>>>  This is not only an issue of "broken sy

Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"

2014-05-23 Thread marc fawzi
I thought .continue/advance was similar to the 'continue' statement in a
for loop in that everything below the statement will be ignored and the
loop would start again from the next index. So my console logging was
giving confusing results. I figured it out and it works fine now. For
sanity's sake, I've resorted to adding a 'return'  in my code in the
.success callback after every .advance and .continue so the execution flow
is easier to follow. It's very confusing, from execution flow perspective,
for execution to continue past .continue/.advance while at once looping
asynchronously. I understand it's two different instances of the .success
callback but it was entirely not clear to me from reading the docs on MDN
(for example) that .advance / .continue are async.

Also, the description of .advance in browser vendor's documentation, e.g.
on MDN, says "Advance the cursor position forward by two places" for
cursor.advance(2) but what they should really say is "advance the cursor
position forward by two results." For example, let's say cursor first
landed on an item with primary key = 7, and you issue the statement
cursor.advance(2), I would expect it to go to the item with primary key 5
(for cursor direction = "prev") but instead it goes to the item with
primary key 2 because that's the 2nd match for the range argument from the
cursor's current position, which means that .advance(n) would be far more
clear semantically speaking if it was simply done as .continue(n)  ... I
guess if there is an understanding that the cursor is always at a matching
item and that it could only continue/advance to the next/prev matching
item, not literal 'positions' in the table (i.e. sequentially through the
list of all items) then there would be no confusion but the very concept of
a cursor is foreign to most front end developers, and that's where the
confusion comes from for many.

My inclination as a front end developer, so far removed from database
terminology, would be

1) to deprecate .advance in favor of .continue(n) and

2) if it makes sense (you have to say why it may not) have
.continue()/.continue(n) cause the return of the execution flow similar to
'continue' in a for loop.

What do you think?



On Wed, May 21, 2014 at 10:42 AM, Joshua Bell  wrote:

>
>
>
> On Wed, May 21, 2014 at 7:32 AM, Arthur Barstow wrote:
>
>> [ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ]
>>
>> On 5/20/14 7:46 PM, marc fawzi wrote:
>>
>>> Hi everyone,
>>>
>>> I've been using IndexedDB for a week or so and I've noticed that
>>> cursor.advance(n) will always move n items forward regardless of cursor
>>> direction. In other words, when the cursor direction is set to "prev" as
>>> in: range = IDBKeyRange.only(someValue, "prev") and primary key is
>>> auto-incremented, the cursor, upon cursor.advance(n), will actually advance
>>> n items in the opposite direction to the cursor.continue() operation.
>>
>>
> That runs contrary to the spec. Both continue() and advance() reference
> the "steps for iterating a cursor" which picks up the direction from the
> cursor object; neither entry point alters the steps to affect the direction.
>
> When you say "you've noticed", are you observing a particular browser's
> implementation or are you interpreting the spec? I did a quick test and
> Chrome, Firefox, and IE all appear to behave as I expected when intermixing
> continue() and advance() calls with direction 'prev' - the cursor always
> moves in the same direction regardless of which call is used.
>
> Can you share sample code that demonstrates the problem, and indicate
> which browser(s) you've tested?
>
>
>
>
>>  This is not only an issue of "broken symmetry" but it presents an
>>> obstacle to doing things like: keeping a record of the primaryKey of the
>>> last found item (after calling cursor.continue for say 200 times) and, long
>>> after the transaction has ended, call our search function again and, upon
>>> finding the same item it found first last time, advance the cursor to the
>>> previously recorded primary key and call cursor.continue 200 times, from
>>> that offset, and repeat whenever you need to fetch the next 200 matching
>>> items. Such algorithm works in the forward direction (from oldest to newest
>>> item) because cursor.advance(n) can be used to position the cursor forward
>>> at the previously recorded primary key (of last found item) but it does not
>>> work in the backward direction (from newest to oldest item) because there
>>> is no way to make the cursor advance backward. It only advances forward,
>>> regardless of its own set direction.
>>>
>>> This example is very rough and arbitrary. But it appears to me that the
>>> cursor.advance needs to obey the cursor's own direction setting. It's
>>> almost like having a car that only moves forward (and can't u-turn) and in
>>> order to move backward you have to reverse the road. That's bonkers.
>>>
>>> What's up with that?
>>>
>>> How naive or terribly misguided a

Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"

2014-05-21 Thread Joshua Bell
On Wed, May 21, 2014 at 7:32 AM, Arthur Barstow wrote:

> [ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ]
>
> On 5/20/14 7:46 PM, marc fawzi wrote:
>
>> Hi everyone,
>>
>> I've been using IndexedDB for a week or so and I've noticed that
>> cursor.advance(n) will always move n items forward regardless of cursor
>> direction. In other words, when the cursor direction is set to "prev" as
>> in: range = IDBKeyRange.only(someValue, "prev") and primary key is
>> auto-incremented, the cursor, upon cursor.advance(n), will actually advance
>> n items in the opposite direction to the cursor.continue() operation.
>
>
That runs contrary to the spec. Both continue() and advance() reference the
"steps for iterating a cursor" which picks up the direction from the cursor
object; neither entry point alters the steps to affect the direction.

When you say "you've noticed", are you observing a particular browser's
implementation or are you interpreting the spec? I did a quick test and
Chrome, Firefox, and IE all appear to behave as I expected when intermixing
continue() and advance() calls with direction 'prev' - the cursor always
moves in the same direction regardless of which call is used.

Can you share sample code that demonstrates the problem, and indicate which
browser(s) you've tested?




> This is not only an issue of "broken symmetry" but it presents an obstacle
>> to doing things like: keeping a record of the primaryKey of the last found
>> item (after calling cursor.continue for say 200 times) and, long after the
>> transaction has ended, call our search function again and, upon finding the
>> same item it found first last time, advance the cursor to the previously
>> recorded primary key and call cursor.continue 200 times, from that offset,
>> and repeat whenever you need to fetch the next 200 matching items. Such
>> algorithm works in the forward direction (from oldest to newest item)
>> because cursor.advance(n) can be used to position the cursor forward at the
>> previously recorded primary key (of last found item) but it does not work
>> in the backward direction (from newest to oldest item) because there is no
>> way to make the cursor advance backward. It only advances forward,
>> regardless of its own set direction.
>>
>> This example is very rough and arbitrary. But it appears to me that the
>> cursor.advance needs to obey the cursor's own direction setting. It's
>> almost like having a car that only moves forward (and can't u-turn) and in
>> order to move backward you have to reverse the road. That's bonkers.
>>
>> What's up with that?
>>
>> How naive or terribly misguided am I being?
>>
>> Thanks in advance.
>>
>> Marc
>>
>
>
>


IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"

2014-05-21 Thread Arthur Barstow

[ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ]

On 5/20/14 7:46 PM, marc fawzi wrote:

Hi everyone,

I've been using IndexedDB for a week or so and I've noticed that 
cursor.advance(n) will always move n items forward regardless of 
cursor direction. In other words, when the cursor direction is set to 
"prev" as in: range = IDBKeyRange.only(someValue, "prev") and primary 
key is auto-incremented, the cursor, upon cursor.advance(n), will 
actually advance n items in the opposite direction to the 
cursor.continue() operation. This is not only an issue of "broken 
symmetry" but it presents an obstacle to doing things like: keeping a 
record of the primaryKey of the last found item (after calling 
cursor.continue for say 200 times) and, long after the transaction has 
ended, call our search function again and, upon finding the same item 
it found first last time, advance the cursor to the previously 
recorded primary key and call cursor.continue 200 times, from that 
offset, and repeat whenever you need to fetch the next 200 matching 
items. Such algorithm works in the forward direction (from oldest to 
newest item) because cursor.advance(n) can be used to position the 
cursor forward at the previously recorded primary key (of last found 
item) but it does not work in the backward direction (from newest to 
oldest item) because there is no way to make the cursor advance 
backward. It only advances forward, regardless of its own set direction.


This example is very rough and arbitrary. But it appears to me that 
the cursor.advance needs to obey the cursor's own direction setting. 
It's almost like having a car that only moves forward (and can't 
u-turn) and in order to move backward you have to reverse the road. 
That's bonkers.


What's up with that?

How naive or terribly misguided am I being?

Thanks in advance.

Marc