Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-20 Thread Leif Hedstrom
On Apr 19, 2024, at 21:42, dave seddon wrote:Thanks Brian! Very cool I don't know if you're supposed to retroactively update release notes, but that's a pretty big feature not in the notes.https://docs.trafficserver.apache.org/en/10.0.x/release-notes/upgrading.en.htmlYeh the release notes for

Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-19 Thread dave seddon
Thanks Brian! Very cool I don't know if you're supposed to retroactively update release notes, but that's a pretty big feature not in the notes. https://docs.trafficserver.apache.org/en/10.0.x/release-notes/upgrading.en.html Stale-while-revaldate feature was a major challenge for some of our

Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-19 Thread Pavel Vazharov
Thank you all for the responses. On Sat, Apr 20, 2024 at 12:02 AM Brian Neradt wrote: > If I understand correctly, ATS does NOT support stale while revalidate, >> which is surprising given the RFC was created by a Yahoo-ligan. ;) >> > > I'm glad you mentioned this. Actually Yahoo open-sourced

Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-19 Thread Brian Neradt
> > If I understand correctly, ATS does NOT support stale while revalidate, > which is surprising given the RFC was created by a Yahoo-ligan. ;) > I'm glad you mentioned this. Actually Yahoo open-sourced the functionality for stale while revalidate via the stale_response.so plugin for ATS 10:

Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-19 Thread dave seddon
Depending on your use case, you also want to carefully consider the cache control headers for the downloaded object. If I understand correctly, ATS does NOT support stale while revalidate, which is surprising given the RFC was created by a Yahoo-ligan. ;) On Fri, Apr 19, 2024 at 9:51 AM Leif

Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-19 Thread Leif Hedstrom
We discussed this in the slack channel. Probably the best option for you (other than writing a new plugin) is to use the background_fetch plugin. You will still need to trigger a download, but you can avoid having to wait for the full response in the client. So, what you’d do is curl -H

Re: [E] Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-17 Thread Walt Karas via users
Seems like it would be dangerous to enable PUSH on a proxy that is accessible to untrusted clients. I guess you could set proxy.local.incoming_ip_to_bind to a loopback address on a second instance of ATS, and use remap rules to access content pushed to it. Could it be that the cache control

Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-17 Thread Pavel Vazharov
Hi Brian, Thank you for your response. It'll do the job but, as far as I understand it, an external functionality will need to download the content in order to push it into the ATS. The content that I want to write in the ATS storage is on the Internet. Regards, Pavel. On Wed, Apr 17, 2024 at

Re: Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-17 Thread Brian Neradt
Hi Pavel, This isn't a direct answer to your question, but are you aware of the ATS HTTP PUSH feature? That allows you to push objects into the ATS cache without the typical caching of proxied response. Can that help you in this situation?

Using Apache Traffic Server as HTTP client to store some content in the storage

2024-04-17 Thread Pavel Vazharov
Hi there, Is there a way to use the ATS as an HTTP client to download and store given content? I'm aware that I can achieve this with a local HTTP client (wget, curl) which uses the ATS as a forward proxy and then the ATS will store the content (which is the actual goal). I was wondering if there