Re: [Mono-dev] mono WebResponse caching

[Daniel Lo Nigro]

>
> It appears that LastModified is always set to the datetime when the
> response comes back from the server


It's up to the server to set the Last-Modified header correctly.
ASP.NETsets it to the current date by default, but pages can override
this.

What I meant with my "nice to have" was cache revalidation using
Last-Modified. To do this, you save the Last-Modified value along with the
cached response. When your cache is expiring, you send a request to the
server, with an "If-Modified-Since" header containing the cached
Last-Modified value. If the server responds with "304 Not Modified", it
means the cached version is still valid, and you can extend its validity
rather than purging it. This can also be done with ETags by using the
"If-None-Match" request header.

Worth noting that for a standards-compliant caching implementation, if the
Cache-Control header contains "must-revalidate", you must send a request
back to the server once the cache expires, with If-Modified-Since
containing the Last-Modified value (and If-None-Match containing the ETag
value). You can only reuse the cached version if the server responds with
"304 Not Modified".

Caching is tricky to get right, would definitely be worth having some unit
tests around this. This page has a fantastic reference on Cache-Control
headers and what they do. Definitely worth reading:
http://www.mnot.net/cache_docs


On Tue, Jun 11, 2013 at 11:27 AM, Mark Lintner wrote:

>  Good ideas. The MemoryCache in System.Runtime.Caching seems interesting
> but it is for our purposes inaccessible because the dll it lives in has  a
> dependency on the System.dll and System.dll already has 3 cross
> dependencies. Still its not impossible. It seems like everything in the
> Cache-Control header is important. That's where max-age it. Ive already got
> no-cache implemented and it looks like max-age=0 means the same thing. It
> appears that LastModified is always set to the datetime when the response
> comes back from the server. You mention that it would be nice to have. Does
> that mean it is not working correctly. I haven't checked whether the value
> is different on Microsoft. I still don't understand how Microsofts web
> cache works. It is persistent in the sense that it is putting the urls in a
> directory. The urls have illegal file names. What is interesting is that
> the speed with which it returns a cached url is roughly equivalent to a
> Dictionary based cache I was testing.
>
>
> --
>
>  *From:* dan...@d15.biz [dan...@d15.biz] on behalf of Daniel Lo Nigro [
> li...@dan.cx]
> *Sent:* Sunday, June 09, 2013 7:36 AM
> *To:* Mark Lintner
> *Cc:* Greg Young; mono-devel-list@lists.ximian.com
>
> *Subject:* Re: [Mono-dev] mono WebResponse caching
>
>   There'd be some code for output caching in Mono's ASP.NETimplementation, I 
> wonder if any of it could be reused here. They're
> different concepts (ASP.NET is using output caching at the server-side
> whereas this is request caching at the client-side) but maybe some of the
> code could be reused, especially around cache header parsing.
>
>  A caching implementation would definitely have to take into account, at
> the minimum:
>
>- Cacheability (public / private)
>- Max age and expiry dates
>
> And revalidation using ETags and Last-Modified dates would be a
> nice-to-have (doing the request with the old ETag and Last-Modified date
> and getting a 304 Not Modified back if the page hasn't been modified) but
> isn't entirely necessary with a basic implementation (don't both
> revalidating, just clear the cache when the max age is met)
>
>  I'd personally cache the results in memory using 
> System.Runtime.Cachingrather
>  than on disk, as on-disk caching can get slow. I haven't checked if
> Mono supports this, though.
>
>
> On Wed, Jun 5, 2013 at 11:32 AM, Mark Lintner wrote:
>
>>  Hi Greg,
>>
>>
>>
>> Great input. I have not gotten that far yet. Also the cache level is very
>> naïve. Everything caches except Bypass cache which doesn't cache. To check
>> the headers it would check the WebResponse Headers property coming
>> into TryInsert in DirectoryBackedResponseCache.cs . I would combine the
>> result of checking the header with the conditional ShouldCache which also
>> needs work.  What we are looking for is comments like this, about what you
>> think needs to be in a usable implementation, gotchas, things to look out
>> for. Also if we are on the wrong track doing it this way I don't want to
>> waste time doing something which will not be useful. Assuming I'm not on
>> the wrong path, I will add the header check next. This is the kind of
>> feedback we need. I only tried to make a prototype of how caching could
>> work and then did some performance measurements. Obviously didn't add any
>> of the details yet. If this is a viable approach I will continue to fill
>> out the implementation guided by research and su

Re: [Mono-dev] mono WebResponse caching

[Mark Lintner]

Good ideas. The MemoryCache in System.Runtime.Caching seems interesting but it 
is for our purposes inaccessible because the dll it lives in has  a dependency 
on the System.dll and System.dll already has 3 cross dependencies. Still its 
not impossible. It seems like everything in the Cache-Control header is 
important. That's where max-age it. Ive already got no-cache implemented and it 
looks like max-age=0 means the same thing. It appears that LastModified is 
always set to the datetime when the response comes back from the server. You 
mention that it would be nice to have. Does that mean it is not working 
correctly. I haven't checked whether the value is different on Microsoft. I 
still don't understand how Microsofts web cache works. It is persistent in the 
sense that it is putting the urls in a directory. The urls have illegal file 
names. What is interesting is that the speed with which it returns a cached url 
is roughly equivalent to a Dictionary based cache I was testing.





From: dan...@d15.biz [dan...@d15.biz] on behalf of Daniel Lo Nigro 
[li...@dan.cx]
Sent: Sunday, June 09, 2013 7:36 AM
To: Mark Lintner
Cc: Greg Young; mono-devel-list@lists.ximian.com
Subject: Re: [Mono-dev] mono WebResponse caching

There'd be some code for output caching in Mono's ASP.NET 
implementation, I wonder if any of it could be reused here. They're different 
concepts (ASP.NET is using output caching at the server-side 
whereas this is request caching at the client-side) but maybe some of the code 
could be reused, especially around cache header parsing.

A caching implementation would definitely have to take into account, at the 
minimum:

  *   Cacheability (public / private)
  *   Max age and expiry dates

And revalidation using ETags and Last-Modified dates would be a nice-to-have 
(doing the request with the old ETag and Last-Modified date and getting a 304 
Not Modified back if the page hasn't been modified) but isn't entirely 
necessary with a basic implementation (don't both revalidating, just clear the 
cache when the max age is met)

I'd personally cache the results in memory using 
System.Runtime.Caching
 rather than on disk, as on-disk caching can get slow. I haven't checked if 
Mono supports this, though.


On Wed, Jun 5, 2013 at 11:32 AM, Mark Lintner 
mailto:mlint...@sinenomine.net>> wrote:

Hi Greg,



Great input. I have not gotten that far yet. Also the cache level is very 
naïve. Everything caches except Bypass cache which doesn't cache. To check the 
headers it would check the WebResponse Headers property coming into TryInsert 
in DirectoryBackedResponseCache.cs . I would combine the result of checking the 
header with the conditional ShouldCache which also needs work.  What we are 
looking for is comments like this, about what you think needs to be in a usable 
implementation, gotchas, things to look out for. Also if we are on the wrong 
track doing it this way I don't want to waste time doing something which will 
not be useful. Assuming I'm not on the wrong path, I will add the header check 
next. This is the kind of feedback we need. I only tried to make a prototype of 
how caching could work and then did some performance measurements. Obviously 
didn't add any of the details yet. If this is a viable approach I will continue 
to fill out the implementation guided by research and suggestions. Probably 
want to limit it to what we really need to have. Any other suggestions, 
comments, criticism would be greatly appreciated.



Thanks,

Mark


From: Greg Young [gregoryyou...@gmail.com]
Sent: Tuesday, June 04, 2013 6:49 PM
To: Mark Lintner
Cc: mono-devel-list@lists.ximian.com
Subject: Re: [Mono-dev] mono WebResponse caching

I would expect this to look at http caching headers in response when caching 
such as expiration, whether something is cachable (public/private) etc but 
don't see any of that code (or maybe I missed it). Can you point to where it is 
in your implementation?

On Tuesday, June 4, 2013, Mark Lintner wrote:

We have noticed that parts of the Mono Web functionality are not yet 
implemented. We are hoping to be able to add some of these missing pieces. 
Caching of

HttpWebResponses is the first one we took a look at. It would appear that 
Microsoft takes advantage of the internet explorer caching. Since it would 
appear

that an HttpWebResponse is fully populated when it is returned from the cache 
and the only content in the urls that are cached are the bytes that come in on

the wire, Microsoft must return things from the cache at a very low level and 
and let the response populate the normal way. This did not seem possible to

emulate when I looked at mono. I also wanted to cache to a directory so that 
the urls can be inspected or delete

Re: [Mono-dev] sub-process invocation on posix

[Greg Najda]

The Windows 
CreateProcessfunction
takes command line arguments as a single string. This detail
leaked into the .NET Process class. Windows programs with a
WinMainentry
point typically break that argument string into arguments using
CommandLineToArgvW.
With a regular "main" entry point, the C runtime does that for you.
Unfortunately there is no ArgvToCommandLine function, which is a shame
because CommandLineToArgvW has pretty funky rules for quotes and
backslashes. See the docs for
CommandLineToArgvWand
Raymond Chen's blog
post for
info. Simply enclosing in quotes and putting a backslash before quotes
and backslashes is not good enough for Windows.

I was curious about this myself a week or two ago because I had to pass
some dynamic arguments to a process so I dove into the Mono source. On
Windows Mono passes the argument string as is to CreateProcess. On Unix
platforms Mono uses the GNOME
g_shell_parse_argv()function
to convert the arg string into an argv before starting the process.

Feel free to use the following code taken from a personal project of
mine.
It passes the unit tests I threw it.

private string QuoteCommandLineArg(string arg)
{
if (Environment.OSVersion.Platform == PlatformID.Unix ||
Environment.OSVersion.Platform == PlatformID.MacOSX)
{
return QuoteCommandLineArgUnix(arg);
}
else
{
return QuoteCommandLineArgWindows(arg);
}
}

internal static string QuoteCommandLineArgWindows(string arg)
{
// If a double quotation mark follows two or an even number of
backslashes,
// each proceeding backslash pair is replaced with one
backslash and the double quotation mark is removed.
// If a double quotation mark follows an odd number of
backslashes, including just one,
// each preceding pair is replaced with one backslash and the
remaining backslash is removed;
// however, in this case the double quotation mark is not
removed.
// -
http://msdn.microsoft.com/en-us/library/system.environment.getcommandlineargs.aspx
//
// Windows command line processing is funky

string escapedArg;
Regex backslashSequenceBeforeQuotes = new Regex(@"(\\+)""");
// Double \ sequences before "s, Replace " with \", double \
sequences at end
escapedArg = backslashSequenceBeforeQuotes.Replace(arg, (match)
=> new string('\\', match.Groups[1].Length * 2) + "\"");
escapedArg = escapedArg.Replace("\"", @"\""");
Regex backslashSequenceAtEnd = new Regex(@"(\\+)$");
escapedArg = backslashSequenceAtEnd.Replace(escapedArg, (match)
=> new string('\\', match.Groups[1].Length * 2));
// C:\blah\"\\
// "C:\blah\\\""
escapedArg = "\"" + escapedArg + "\"";
return escapedArg;
}

internal static string QuoteCommandLineArgUnix(string arg)
{
// Mono uses the GNOME g_shell_parse_argv() function to convert
the arg string into an argv
// Just prepend " and \ with \ and enclose in quotes.
// Much simpler than Windows!

Regex backslashOrQuote = new Regex(@"\\|""");
return "\"" + backslashOrQuote.Replace(arg, (match) => @"\" +
match.ToString()) + "\"";
}


Hope that helps.

- Greg


On Mon, Jun 10, 2013 at 3:46 PM, Ian Norton  wrote:

> I kind of already have a thing to do that, feels a bit icky though,
> especially as there must be some thing lower down that undoes the joined up
> string into a char** again. :)
>
>
> On 10 June 2013 16:06, Michael Hutchinson wrote:
>
>> FWIW, you actually just need to double quote each argument and escape
>> double quotes so you can very easily write a helper to do this in a way
>> that works on both Mono and .NET:
>>
>> static Process StartProcess (string name, params string[] args)
>> {
>> string a = null;
>> if (args != null && args.Length > 0)
>> a = "\"" + string.Join (args.Select (s => s.Replace ("\"",
>> "\\\"")).ToArray (), "\" \"") + "\"";
>> return Process.Start (
>> new ProcessStartInfo (name, a) {
>> ShellExecute = false,
>> }
>> );
>> }
>>
>> Obviously this could be done more efficiently with a S

Re: [Mono-dev] sub-process invocation on posix

[Ian Norton]

I kind of already have a thing to do that, feels a bit icky though,
especially as there must be some thing lower down that undoes the joined up
string into a char** again. :)


On 10 June 2013 16:06, Michael Hutchinson  wrote:

> FWIW, you actually just need to double quote each argument and escape
> double quotes so you can very easily write a helper to do this in a way
> that works on both Mono and .NET:
>
> static Process StartProcess (string name, params string[] args)
> {
> string a = null;
> if (args != null && args.Length > 0)
> a = "\"" + string.Join (args.Select (s => s.Replace ("\"",
> "\\\"")).ToArray (), "\" \"") + "\"";
> return Process.Start (
> new ProcessStartInfo (name, a) {
> ShellExecute = false,
> }
> );
> }
>
> Obviously this could be done more efficiently with a StringBuilder.
>
> Apologies for any errors, I'm writing this on my phone...
>
> - Michael
> On Jun 6, 2013 1:18 PM, "Ian Norton"  wrote:
>
>> Hiya, I'm aware that I can use Process.Start() but I'd really really like
>> to be able to directly pass a list of strings to my child process as
>> arguments rather than having to escape shell characters and spaces etc.
>>
>> Ie, In perl or C I'd do:
>>
>> system("df","-m","/home/foo/Documents/Pictures/My Holiday");
>>
>> Where in c# I'm forced to escape the space -> "My\ Holiday"
>>
>> Ian
>>
>> ___
>> Mono-devel-list mailing list
>> Mono-devel-list@lists.ximian.com
>> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>>
>>
___
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list

Re: [Mono-dev] sub-process invocation on posix

[Michael Hutchinson]

FWIW, you actually just need to double quote each argument and escape
double quotes so you can very easily write a helper to do this in a way
that works on both Mono and .NET:

static Process StartProcess (string name, params string[] args)
{
string a = null;
if (args != null && args.Length > 0)
a = "\"" + string.Join (args.Select (s => s.Replace ("\"",
"\\\"")).ToArray (), "\" \"") + "\"";
return Process.Start (
new ProcessStartInfo (name, a) {
ShellExecute = false,
}
);
}

Obviously this could be done more efficiently with a StringBuilder.

Apologies for any errors, I'm writing this on my phone...

- Michael
On Jun 6, 2013 1:18 PM, "Ian Norton"  wrote:

> Hiya, I'm aware that I can use Process.Start() but I'd really really like
> to be able to directly pass a list of strings to my child process as
> arguments rather than having to escape shell characters and spaces etc.
>
> Ie, In perl or C I'd do:
>
> system("df","-m","/home/foo/Documents/Pictures/My Holiday");
>
> Where in c# I'm forced to escape the space -> "My\ Holiday"
>
> Ian
>
> ___
> Mono-devel-list mailing list
> Mono-devel-list@lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
>
___
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list