Thanks for your answers. I've implemented an Input representation more or less
like the code you sent. each time a new "chunk" or url is required, I scan the
Ranges first and if the ranges don't require data from that chunk, I just use a
dummy input stream to pass zeros. It's not perfect (perfect would not require
passing zeros), but in my cursory testing so far it works reasonably well, and
I suspect it will be more than adequate.
bjorn
On Apr 20, 2012, at 6:25 AM, Thierry Boileau wrote:
> Hello Bjorn,
>
> - Are you suggesting that I return an InputRepresentation instead of an
> OutputRepresentation? That seems counterintuitive, it seems like I should
> always be returning some sort of OutputRepresentation because I am outputing
> data to the client. Terminology wise, my intuition tells me an input
> representation would be for data coming in from the client. I don't see this
> distinction documented anywhere, though.
> I think that in your case the approach based on InputRepresentation is
> better, and should offer better performance.
> The aim of representations based on streams (such as InputRepresentation or
> OutputRepresentation) is to deliver the bytes of the Representation that are
> meant to be read (then written on a socket for example) via the stream
> returned by the "getStream()"method. InputRepresentation delivers the stream
> via an input stream, OutputRepresentation via output stream.
> Please feel free to ask for more details.
>
> - I am still confused as to what the bug was, and just what,exactly, I'm
> working around.
> I agree also, but it is linked to the usage of the OutputRepresentation.
>
> - The reason I don't want to use one of the "canned" or "pre-joined"
> sequences is performance: why am I reading 10 URLs off if the user requested
> data from only one of them? Ultimately, my goal is not not have to read data
> that is going to be dumped anyway.
>
> - I still don't see how to bypass the content range stuff. The docs allude to
> this being possible in several places, so I kindof assumed it could be done,
> but I don't see it documented explicitly, so maybe not. At the very least, of
> course, I can read the range info, determine what will not be read and fill
> in zeros for that information. That approach is still a lot better than
> reading all the data from a remote URL.
> I've not tested it, I wonder if the RangeFilter really consumes the whole
> Representation. If this is the case, I think we should do better. I check
> that today.
> Regarding the usage of the "pre-joined" sequences, you can still check if the
> streams are really opened and read (I think the one from Guava should perform
> correctly, if not, you can complete the one I sent to you).
>
> Best regards,
> Thierry Boileau
>
>
> bjorn
>
> On Apr 18, 2012, at 12:07 PM, Thierry Boileau wrote:
>
>> Hello Bjorn,
>>
>> >By messing with the code you sent, this seems to be a function of setting
>> >the representation size. If I set the size, it's fine, if not, it sends the
>> >whole thing regardless of content-range, which seems like a bug.
>> Thanks for discovering a bug inside the RangeFilter class.
>>
>> Then, I've found that the problem is due to the usage of the
>> OutputRepresentation in order to gather the several instances of
>> InputStream. I propose you to use an InputRepresentation instead.
>> First, I've set up a InputStream class that composes the several InputStream
>> (this implementation is only a "quick" code):
>>
>> private static class CompositeInputStream extends InputStream {
>> private Iterator<URL> urlIt = null;
>> private InputStream is = null;
>>
>> public CompositeInputStream(Vector<URL> urls) throws IOException {
>> urlIt = urls.iterator();
>> if (urlIt.hasNext()) {
>> is = urlIt.next().openStream();
>> }
>> }
>>
>> @Override
>> public int read() throws IOException {
>> // read each input stream until exhaustion
>> int r = is.read();
>>
>> boolean goOn = (r == -1);
>> while (goOn) {
>> // choose the next one until exhaustion
>> if (urlIt.hasNext()) {
>> is = urlIt.next().openStream();
>> r = is.read();
>> goOn = (r == -1);
>> } else {
>> goOn = false;
>> }
>> }
>>
>> return r;
>> }
>> }
>>
>> Then, I simple return the InputRepresentation:
>> return new InputRepresentation(new CompositeInputStream(urls));
>>
>> I hope this will help you.
>>
>> Best regards,
>> Thierry Boileau
>>
>>
>>
>> On Apr 11, 2012, at 2:14 PM, Bjorn Roche wrote:
>>
>>>
>>> On Apr 11, 2012, at 1:50 PM, Thierry Boileau wrote:
>>>
>>>> Hi Bjorn,
>>>>
>>>> that's right, I primarily use the restlet 2.1 version, which does not
>>>> support this constructor I think. I'm not sure this has an impact.
>>>>
>>>
>>> Well when I use the setter I get the same thing (throws exception) as when
>>> I pass that value to the constructor.
>>>
>>> Also, it looks like the range is being ignored -- I am getting the full
>>> document back rather than part.
>>
>> By messing with the code you sent, this seems to be a function of setting
>> the representation size. If I set the size, it's fine, if not, it sends the
>> whole thing regardless of content-range, which seems like a bug.
>>
>>> Can you send me build instructions (pom or ant file?) for your code if you
>>> have it? I am testing with curl, maybe there is an issue with one of our
>>> tests?
>>
>> I've figured out how to build (I am an idiot with building)... and I can't
>> reproduce the exception.
>>
>> bjorn
>>
>>>
>>> Finally, is it possible to avoid the range processing on just some calls
>>> for greater efficiency?
>>>
>>> bjorn
>>>
>>>> Best regards,
>>>> Thierry Boileau
>>>>
>>>> Thierry,
>>>>
>>>> The first thing I noticed in your code is that you are using the
>>>> OutputRepresentation constructor that does not take an expectedSize
>>>> argument, so I took that out and the exception goes away. At the moment my
>>>> tests are still failing, so I'll look into that some more.
>>>>
>>>> bjorn
>>>>
>>>> On Apr 11, 2012, at 12:39 PM, Thierry Boileau wrote:
>>>>
>>>>> Hi Bjorn,
>>>>>
>>>>> I send you a sample test code (server + client) based on your code that
>>>>> works for me. But I notice that my app sends only 15000 bytes... Can you
>>>>> tell us the metrics of your tests?
>>>>>
>>>>> Not 100% sure of the question.. do you mean container? I am not using
>>>>> jetting. My initialization code looks like this:
>>>>> Component component = new Component();
>>>>> component.getServers().add(Protocol.HTTP, PORT);
>>>>> component.getDefaultHost().attach(new
>>>>> com.xonami.rest.server.ApiApplication());
>>>>> component.start();
>>>>>
>>>>> Is that what you wanted?
>>>>> I wanted to know if the application is served using some "server"
>>>>> extensions such as jetty (org.restlet.ext.jetty.jar) or simple
>>>>> (org.restlet.ext.simple.jar), or inside a servlet container.
>>>>>
>>>>>
>>>>> also, in the code I sent it looks like it was not "hanging" but rather
>>>>> pausing for a moment and then throwing this exception:
>>>>>
>>>>> java.io.IOException: Timeout while writing to the queue-based output
>>>>> stream
>>>>> at org.restlet.engine.io.PipeStream$2.write(PipeStream.java:106)
>>>>> at java.io.OutputStream.write(OutputStream.java:99)
>>>>> at
>>>>> com.xonami.rest.server.media.RawMediaStateWithIdResource$1.write(RawMediaStateWithIdResource.java:158)
>>>>> at org.restlet.engine.io.BioUtils$2.run(BioUtils.java:394)
>>>>> at org.restlet.service.TaskService$1$1.run(TaskService.java:130)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:680)
>>>>>
>>>>> thanks, I try to reproduce this error. Or, if may ask you, can you
>>>>> provide a reproductible sample test code?
>>>>>
>>>>> Best regards,
>>>>> Thierry Boileau
>>>>>
>>>>> -----------------------------
>>>>> Bjorn Roche
>>>>> http://www.xonami.com
>>>>> Audio Collaboration
>>>>> http://blog.bjornroche.com
>>>>>
>>>>> ------------------------------------------------------
>>>>> http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2947368
>>>>>
>>>>> <testRange.zip>
>>>>
>>>>
>>>> -----------------------------
>>>> Bjorn Roche
>>>> http://www.xonami.com
>>>> Audio Collaboration
>>>> http://blog.bjornroche.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> -----------------------------
>>> Bjorn Roche
>>> http://www.xonami.com
>>> Audio Collaboration
>>> http://blog.bjornroche.com
>>>
>>>
>>>
>>>
>>
>>
>> -----------------------------
>> Bjorn Roche
>> http://www.xonami.com
>> Audio Collaboration
>> http://blog.bjornroche.com
>>
>>
>>
>>
>>
>
> -----------------------------
> Bjorn Roche
> http://www.xonami.com
> Audio Collaboration
> http://blog.bjornroche.com
>
>
>
>
>
-----------------------------
Bjorn Roche
http://www.xonami.com
Audio Collaboration
http://blog.bjornroche.com
------------------------------------------------------
http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2950205
