[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512242#comment-15512242 ] Ebrahim Mohammadi commented on TS-974: -- Is there a plan for merging the code? In which version do you guess it'll be released? > TS should have a mode to hold partial objects in cache > -- > > Key: TS-974 > URL: https://issues.apache.org/jira/browse/TS-974 > Project: Traffic Server > Issue Type: Improvement > Components: Cache >Affects Versions: 3.0.1 >Reporter: William Bardwell >Assignee: Alan M. Carroll > Labels: A > Fix For: 7.1.0 > > > For ATS to do an excelent job caching large files like video it would need to > be able to hold partial objects for a large file. This could be done in a > plugin or in the core. This would need to be integrated with the Range > handling code to serve requests out of the partial objects and to get more > parts of a file to satisfy a Range request. > An intermediate step (also do-able in the core or in a plugin) would be to > have some settings to let the Range handling code be able to trigger a full > file download either asynchronously when a Range response indicates that the > file isn't larger than some threshold, or synchronously when a Range request > could reasonably be answered quickly from a full request. (Right now Range > requests are tunneled if there is not full cached content as far as I can > tell.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632814#comment-14632814 ] ASF subversion and git services commented on TS-974: Commit 1c06db83123c336d462083002eda9ffcd4852730 in trafficserver's branch refs/heads/poc-6-0-x from [~amc] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=1c06db8 ] TS-974: Partial Object Caching. TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell Assignee: Alan M. Carroll Labels: A Fix For: 6.1.0 For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605500#comment-14605500 ] ASF subversion and git services commented on TS-974: Commit 528eab64a26e869ce69a4bb3729c7441cd9b4906 in trafficserver's branch refs/heads/poc-6-0-x from [~amc] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=528eab6 ] TS-974: Partial Object Caching. TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell Assignee: Alan M. Carroll Labels: A Fix For: 6.1.0 For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502021#comment-14502021 ] ASF subversion and git services commented on TS-974: Commit 799a83bb4bdf8a34b5716289e6b9e2edc60772cf in trafficserver's branch refs/heads/ts-974-5-3-x from [~amc] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=799a83b ] TS-974: Base implmentation. TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell Assignee: Alan M. Carroll Labels: A Fix For: 6.0.0 For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502166#comment-14502166 ] ASF subversion and git services commented on TS-974: Commit f82fd704fc0037968e340a844114a703cd50f3a5 in trafficserver's branch refs/heads/ts-974-5-3-x from [~amc] [ https://git-wip-us.apache.org/repos/asf?p=trafficserver.git;h=f82fd70 ] TS-974: Additional fixes. Cherry-pick from 5.2.x version. TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell Assignee: Alan M. Carroll Labels: A Fix For: 6.0.0 For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196518#comment-14196518 ] Alan M. Carroll commented on TS-974: This is now under active development. TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell Assignee: Alan M. Carroll Labels: A Fix For: 5.3.0 For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617435#comment-13617435 ] Leif Hedstrom commented on TS-974: -- I had some discussions on this topic with john, and a few ideas perculated: 1) The old idea I posted before, about doing background fills of the entire objects, and then rely on read-while-writer to serve partial requests while filling the cache. This is by far the easiest solution. 2) An interesting idea is to do a plugin (and perhaps a few core changes) such that we simply cache each segment size of a URL as an individual object. As an example, if the URL http://example.com/foo.m4v is served via Range: requests only, we create n cache objects, each with a cache key of (e.g.) http://example.com/foo.mv4-m, where n is the segment number 1-n. So, http://example.com/foo.mv4-1, http://example.com/foo.mv4-2, http://example.com/foo.mv4-3 etc. We then have plugins (and/or core) components stitch / filter responses back by simply finding the appropriate segments with the well defined URLs. 3) Do the partial writes to disk, like bittorrent does. #2 is interesting to me, since it requires no changes to the cache. It can get a bit complicated though, since the stitching transforms have to access some arbitrary amount of cache entries. But presumably, we can satisfy many or all requests with 1 or a small number of cached objects. In addition, we could combine this approach with #1, such that when #1 is running and requests can be satisfied out of the full down load, we don't have to do #2 (so #2 is only a stopgap measurement to allow partial caching when users are fetching truly random pieces out of an uncached object). Thoughts? TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell Fix For: sometime For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617449#comment-13617449 ] William Bardwell commented on TS-974: - #1 is certainly a reasonable step and not too hard, just needs some configuration about when to trigger it, and when to quit. (And also configuring, do you need to sometimes tunnel the range request while doing the background full load.) #2 is also reasonable, as you say the hard part is doing proper stitching together segments when the range request covers several of them, and ideally you could get at the new pread cache support to avoid reading too much data that you have no interest in. #2 does have some other benefits, like that it allows parts of files to be never cached, or expired if you have a video that is too boring to actually watch all of it, or I have heard that some pdf viewers look at the beginning and the end of a file with range requests (before they look at the middle), and that working without having to load the whole file first would be appealing. #3 would be awesome, but sounds hard. TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell Fix For: sometime For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271664#comment-13271664 ] James Peach commented on TS-974: Related to TS-475. If we are lucky TS-475 will fix this too. TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-974) TS should have a mode to hold partial objects in cache
[ https://issues.apache.org/jira/browse/TS-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13197819#comment-13197819 ] Johan Acevedo commented on TS-974: -- Is there currently any cache software for http that manages to do this? TS should have a mode to hold partial objects in cache -- Key: TS-974 URL: https://issues.apache.org/jira/browse/TS-974 Project: Traffic Server Issue Type: Improvement Components: Cache Affects Versions: 3.0.1 Reporter: William Bardwell For ATS to do an excelent job caching large files like video it would need to be able to hold partial objects for a large file. This could be done in a plugin or in the core. This would need to be integrated with the Range handling code to serve requests out of the partial objects and to get more parts of a file to satisfy a Range request. An intermediate step (also do-able in the core or in a plugin) would be to have some settings to let the Range handling code be able to trigger a full file download either asynchronously when a Range response indicates that the file isn't larger than some threshold, or synchronously when a Range request could reasonably be answered quickly from a full request. (Right now Range requests are tunneled if there is not full cached content as far as I can tell.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
