Re: [Bug-wget] Add LIFO queue option for recursive download

[Tim Ruehsen]

On Monday 05 January 2015 19:06:37 John Peterson wrote:
> ### enqueue html last isnt enough
> 
> 
> >This is not what I said. I said: enqueue html last + enqueue non-html
> >first
> 
> 
> > This basically the same as having two queues: one for HTML and one for
> > non-HTML. non-HTML working as LIFO, always picked before HTML. If empty,
> > pick from HTML queue (FIFO).
> 
> show it with code because i dont understand
> 
> the current FIFO code is:
> 
>     while (1)
>         // FIFO
>         url_dequeue
> 
>         if (descend)
>             for (; child; child = child->next)
>                 url_enqueue
>                 
> the LIFO solution is:
> 
>     while (1)
>         // LIFO
>         url_dequeue
> 
>         if (descend)
>             // place html pages on top
>             ll_bubblesort(&child);
>             for (; child; child = child->next)
>                 url_enqueue

I suggest
    while (1)
        url_dequeue

        if (descend)
            for (; child; child = child->next)
                if child is HTML
                  url_enqueue_append
                else
                  url_enqueue_prepend

This results in this download order:
i.html
  y.jpg
  x.jpg
a.html
  a-y.jpg
  a-x.jpg
b.html
  b-y.jpg
  b-x.jpg
a-a.html
  a-a-y.jpg
  a-a-x.jpg
a-b.html
  a-b-y.jpg
  a-b-x.jpg
b-a.html
  b-a-y.jpg
  b-a-x.jpg
b-b.html
  b-b-y.jpg
  b-b-x.jpg

If you want to try this order, I could create a patch for testing.

BTW, since you are talking about timing, people with different bandwidths 
will/might experience different results when viewing such a page. I use mobile 
internet with 2G/3G <= 64kbit/s. I wonder how that works ;-)

Tim

signature.asc
Description: This is a digitally signed message part.