Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
On 2 April 2012 22:25, Stuart Dallas stu...@3ft9.com wrote: On 2 Apr 2012, at 15:37, Simon wrote: On 2 April 2012 14:27, Stuart Dallas stu...@3ft9.com wrote: On 2 Apr 2012, at 14:12, Simon wrote: Thanks Maciek On 2 April 2012 10:37, Maciek Sokolewicz maciek.sokolew...@gmail.com wrote: On 02-04-2012 10:12, Simon wrote: Thanks Simon. you got my hopes up there for a second. From the php docs page: Critics further argue that it is pointless to use a Singleton in a Shared Nothing Architecture like PHP where objects are uniquewithin the Request only anyways. I want the the singleton class to be global to the entire application (ie shared by reference across all requests). I'd agree with the above critics that if you have to instantiate your singleton for each request, it's rather pointless. Well, that's simply not possible due to the shared nothing paradigm. If you want to share, you need to either share it via another medium (such as a database, as has been suggested a dozen times already) or switch to a different language. PHP is based on this paradigm, and you should not expect of it to violate it just because you want to do things a certain way, which is not the PHP way. The existence of memcached, shm and apc_fetch tell me that PHP already accepts the need for sharing data between processes. All I'm arguing for is the ability to share the data by reference rather than by copy. As already mentioned several times the closest you will get is shared memory (as used by APC), but you can't access that by reference because shared read/write resources need controlled access for stability. I know. I understand that (and the issues with locking that might arise if truly shared memory was available). I can't find any material that explains how the .net framework implements application variables. You mentioned earlier that you *know* that when you access them you do so by reference. Do you have a source for this knowledge or is it some sort of sixth sense? Source: 10+ years as an ASP and ASP.NET developer. Wow. As knowledge goes that's up there with I believe it therefore it is. I don't understand your point. Having looked for documentation, I agree, it's utterly terrible. It's as if even Microsoft themselves don't fully understand the advantages that application variables give them over the competition. (Though they're hardly likely to be forthcoming about helping others implement similar features). Here's some stuff I did find: http://www.codeproject.com/Articles/87316/A-walkthrough-to-Application-State#e This article explains basically how application variables work. It doesn't specifically mention passing by reference but it discusses thread safety at some length so you might infer that implies passing by reference. can cause performance overhead if it is heavy You certainly have to be careful how you use it (as you do any tool). | And you want to store petabytes of data in there. Smart. No, I want to store 50 - 200Mb. At some point in the somewhat distant future I can imagine larger amounts of shared storage between required. Up to and including petabytes of data when such large amounts of RAM become practical. This does not have to cause overhead. The author was suggesting caution to protect people who aren't sure what they're doing. This can be done with no performance overhead (so long as you have the physical RAM). Anyway, that page says nothing about whether it's accessed by reference. It implies that writes are atomic, which in turn implies that there is something in the background controlling write access to the data. As for reading the data being done via references there is nothing in that article to suggest that. http://msdn.microsoft.com/en-us/library/ff650849.aspx Here is a more technical discussion about the singleton class is implemented in .NET. Application variables are provided by an instance of a singleton class (the HttpApplication class). Looks no different to the way you'd implement a singleton in PHP (as expected - as a concept it doesn't generally change between languages). Just because the HttpApplication class is a singleton doesn't mean that when you request data from it you get it by reference. Could we just for the sake of argument, just accept that it does actually pass by reference between threads, but not between processes ? It does - is that really so hard to believe ? http://stackoverflow.com/questions/1132373/do-asp-net-application-variables-pass-by-reference-or-value Stack overflow question from someone actually wanting to get a *copy* of an application variable rather than a reference. According to that page scalar values are returned by value, objects are returned as a copy of the reference. Based on that, it would appear that you are correct as far as objects
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
2012/4/1 Simon slg...@gmail.com Another thing that's possible in .NET is the Singleton design pattern. (Application variables are an implementation of this pattern) This makes it possible to instantiate a static class so that a single instance of the object is available to all threads (ie requests) across your application. So for example the code below creates a single instance of an object for the entire server. Any code calling new App(); gets a pointer to the shared object. If PHP could do this, it would be *awesome* and I wouldn't need application variables since this is a superior solution. Can / could PHP do anything like this ? public class App { private static App instance; private App() {} public static App Instance { get { if (instance == null) { instance = new App(); } return instance; } } } Creates an inste Hi, Simon Sorry for this out-of-context post - but just to answer to Simon's question: One way of implementing Singleton in PHP is written down in the php-manual: http://www.php.net/manual/en/language.oop5.patterns.php I personally would also declare __clone() and __wakeup() as private, but that's something personal :) If you have many places where you'd like to use the Singleton-pattern you may now think of having one class where you define the pattern itself and extending other classes from that ... But that does not seem the good way to me because this classes are not related at all, in case of content. Other frameworks are using interfaces but they have to write the code for the implementation over and over again. Here's where traids make the most sense to me: http://stackoverflow.com/questions/7104957/building-a-singleton-trait-with-php-5-4 Bye Simon -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
Thanks Simon. you got my hopes up there for a second. From the php docs page: Critics further argue that it is pointless to use a Singleton in a Shared Nothing Architecture like PHP where objects are unique within the Request only anyways. I want the the singleton class to be global to the entire application (ie shared by reference across all requests). I'd agree with the above critics that if you have to instantiate your singleton for each request, it's rather pointless. On 2 April 2012 07:54, Simon Schick simonsimc...@googlemail.com wrote: 2012/4/1 Simon slg...@gmail.com Another thing that's possible in .NET is the Singleton design pattern. (Application variables are an implementation of this pattern) This makes it possible to instantiate a static class so that a single instance of the object is available to all threads (ie requests) across your application. So for example the code below creates a single instance of an object for the entire server. Any code calling new App(); gets a pointer to the shared object. If PHP could do this, it would be *awesome* and I wouldn't need application variables since this is a superior solution. Can / could PHP do anything like this ? public class App { private static App instance; private App() {} public static App Instance { get { if (instance == null) { instance = new App(); } return instance; } } } Creates an inste Hi, Simon Sorry for this out-of-context post - but just to answer to Simon's question: One way of implementing Singleton in PHP is written down in the php-manual: http://www.php.net/manual/en/language.oop5.patterns.php I personally would also declare __clone() and __wakeup() as private, but that's something personal :) If you have many places where you'd like to use the Singleton-pattern you may now think of having one class where you define the pattern itself and extending other classes from that ... But that does not seem the good way to me because this classes are not related at all, in case of content. Other frameworks are using interfaces but they have to write the code for the implementation over and over again. Here's where traids make the most sense to me: http://stackoverflow.com/questions/7104957/building-a-singleton-trait-with-php-5-4 Bye Simon
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
On 02-04-2012 10:12, Simon wrote: Thanks Simon. you got my hopes up there for a second. From the php docs page: Critics further argue that it is pointless to use a Singleton in a Shared Nothing Architecture like PHP where objects are uniquewithin the Request only anyways. I want the the singleton class to be global to the entire application (ie shared by reference across all requests). I'd agree with the above critics that if you have to instantiate your singleton for each request, it's rather pointless. Well, that's simply not possible due to the shared nothing paradigm. If you want to share, you need to either share it via another medium (such as a database, as has been suggested a dozen times already) or switch to a different language. PHP is based on this paradigm, and you should not expect of it to violate it just because you want to do things a certain way, which is not the PHP way. - Tul -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
Thanks Maciek On 2 April 2012 10:37, Maciek Sokolewicz maciek.sokolew...@gmail.comwrote: On 02-04-2012 10:12, Simon wrote: Thanks Simon. you got my hopes up there for a second. From the php docs page: Critics further argue that it is pointless to use a Singleton in a Shared Nothing Architecture like PHP where objects are uniquewithin the Request only anyways. I want the the singleton class to be global to the entire application (ie shared by reference across all requests). I'd agree with the above critics that if you have to instantiate your singleton for each request, it's rather pointless. Well, that's simply not possible due to the shared nothing paradigm. If you want to share, you need to either share it via another medium (such as a database, as has been suggested a dozen times already) or switch to a different language. PHP is based on this paradigm, and you should not expect of it to violate it just because you want to do things a certain way, which is not the PHP way. The existence of memcached, shm and apc_fetch tell me that PHP already accepts the need for sharing data between processes. All I'm arguing for is the ability to share the data by reference rather than by copy. - Tul
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
On 2 Apr 2012, at 14:12, Simon wrote: Thanks Maciek On 2 April 2012 10:37, Maciek Sokolewicz maciek.sokolew...@gmail.comwrote: On 02-04-2012 10:12, Simon wrote: Thanks Simon. you got my hopes up there for a second. From the php docs page: Critics further argue that it is pointless to use a Singleton in a Shared Nothing Architecture like PHP where objects are uniquewithin the Request only anyways. I want the the singleton class to be global to the entire application (ie shared by reference across all requests). I'd agree with the above critics that if you have to instantiate your singleton for each request, it's rather pointless. Well, that's simply not possible due to the shared nothing paradigm. If you want to share, you need to either share it via another medium (such as a database, as has been suggested a dozen times already) or switch to a different language. PHP is based on this paradigm, and you should not expect of it to violate it just because you want to do things a certain way, which is not the PHP way. The existence of memcached, shm and apc_fetch tell me that PHP already accepts the need for sharing data between processes. All I'm arguing for is the ability to share the data by reference rather than by copy. As already mentioned several times the closest you will get is shared memory (as used by APC), but you can't access that by reference because shared read/write resources need controlled access for stability. I can't find any material that explains how the .net framework implements application variables. You mentioned earlier that you *know* that when you access them you do so by reference. Do you have a source for this knowledge or is it some sort of sixth sense? I would not feel comfortable having my code creating multiple pointers to a read/write segment of shared memory that then uncontrollably float around umpteen processes. If MS have something akin to this in .net I would be extremely interested in reading about how it works without imploding. -Stuart -- Stuart Dallas 3ft9 Ltd http://3ft9.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
On 2 Apr 2012, at 15:37, Simon wrote: On 2 April 2012 14:27, Stuart Dallas stu...@3ft9.com wrote: On 2 Apr 2012, at 14:12, Simon wrote: Thanks Maciek On 2 April 2012 10:37, Maciek Sokolewicz maciek.sokolew...@gmail.comwrote: On 02-04-2012 10:12, Simon wrote: Thanks Simon. you got my hopes up there for a second. From the php docs page: Critics further argue that it is pointless to use a Singleton in a Shared Nothing Architecture like PHP where objects are uniquewithin the Request only anyways. I want the the singleton class to be global to the entire application (ie shared by reference across all requests). I'd agree with the above critics that if you have to instantiate your singleton for each request, it's rather pointless. Well, that's simply not possible due to the shared nothing paradigm. If you want to share, you need to either share it via another medium (such as a database, as has been suggested a dozen times already) or switch to a different language. PHP is based on this paradigm, and you should not expect of it to violate it just because you want to do things a certain way, which is not the PHP way. The existence of memcached, shm and apc_fetch tell me that PHP already accepts the need for sharing data between processes. All I'm arguing for is the ability to share the data by reference rather than by copy. As already mentioned several times the closest you will get is shared memory (as used by APC), but you can't access that by reference because shared read/write resources need controlled access for stability. I know. I understand that (and the issues with locking that might arise if truly shared memory was available). I can't find any material that explains how the .net framework implements application variables. You mentioned earlier that you *know* that when you access them you do so by reference. Do you have a source for this knowledge or is it some sort of sixth sense? Source: 10+ years as an ASP and ASP.NET developer. Wow. As knowledge goes that's up there with I believe it therefore it is. Having looked for documentation, I agree, it's utterly terrible. It's as if even Microsoft themselves don't fully understand the advantages that application variables give them over the competition. (Though they're hardly likely to be forthcoming about helping others implement similar features). Here's some stuff I did find: http://www.codeproject.com/Articles/87316/A-walkthrough-to-Application-State#e This article explains basically how application variables work. It doesn't specifically mention passing by reference but it discusses thread safety at some length so you might infer that implies passing by reference. can cause performance overhead if it is heavy And you want to store petabytes of data in there. Smart. Anyway, that page says nothing about whether it's accessed by reference. It implies that writes are atomic, which in turn implies that there is something in the background controlling write access to the data. As for reading the data being done via references there is nothing in that article to suggest that. http://msdn.microsoft.com/en-us/library/ff650849.aspx Here is a more technical discussion about the singleton class is implemented in .NET. Application variables are provided by an instance of a singleton class (the HttpApplication class). Looks no different to the way you'd implement a singleton in PHP (as expected - as a concept it doesn't generally change between languages). Just because the HttpApplication class is a singleton doesn't mean that when you request data from it you get it by reference. http://stackoverflow.com/questions/1132373/do-asp-net-application-variables-pass-by-reference-or-value Stack overflow question from someone actually wanting to get a *copy* of an application variable rather than a reference. According to that page scalar values are returned by value, objects are returned as a copy of the reference. Based on that, it would appear that you are correct as far as objects are concerned. However, the advice given on several pages I've found, including at least one that you've specified, is to use application variables lightly and sparingly. It's not something that should be used for petabytes of data, mainly because it's all STORED IN MEMORY. So, if you read enough of this stuff, you'll find out that a .NET website (an IIS application in MS parlance) runs in a multi-threaded single process. Application variables (and singleton classes) are shared by reference between all threads. It is possible to run an IIS application over multiple processes (to take advantage of multiple processors / server farms for example) but then the Application variables are not shared at all. (This is then pretty comparable to the situation with node.js except that node is not multi-threaded) Single process. Yes.
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
On 1 April 2012 13:52, Simon slg...@gmail.com wrote: On 31 March 2012 20:44, Stuart Dallas stu...@3ft9.com wrote: On 31 Mar 2012, at 13:14, Simon wrote: Thanks again Stuart. On 31 March 2012 12:50, Stuart Dallas stu...@3ft9.com wrote: On 31 March 2012 11:19, Simon slg...@gmail.com wrote: Thanks for your answer. On 31 March 2012 09:50, Stuart Dallas stu...@3ft9.com wrote: On 31 Mar 2012, at 02:33, Simon wrote: Or: Why doesn't PHP have Applications variables like ASP.NET (and node.js) ? Hi, I'm working on optimising a php application (Drupal). The best optimisation I've found so far is to use APC to store various bits of Drupal data in RAM. The problem with this is that with Drupal requiring say 50Mb of data* per request is that lots of cpu cycles are wasted de-serialising data out of apc_fetch. Also 50Mb of data per http process !! is wasted by each one re-creating it's own copy of the shared data. 50MB? WTF is it storing?? I've never used Drupal, but based purely on that it sounds like an extremely inefficient piece of software that's best avoided! All sorts of stuff (taxonomies, lists of data, menu structures, configuration settings, content etc). Drupal is a sophisticated application. Besides, 50Mb of data seems like relatively tiny application state to want to access in fastest possible way. It's not hard to imagine wanting to use *much* more than this in future If it were possible for apc_fetch (or similar function) to return a pointer to the data rather than a copy of the data this would enable incredible reduction in cpu and memory usage. Vanilla PHP adheres to a principle known as shared nothing architecture in which, shockingly, nothing is shared between processes or requests. This is primarily for scalability reasons; if you stick to the shared nothing approach your application should be easily scalable. Yes, I know. I think the effect of this is that php will scale better (on average) in situations where requests don't need to share much data such as shared hosting. In an enterprise enviroment where the whole server might be dedicated to single application, shared nothing seems to be a synonym for re-load everything ? Yes, on one level that is what it means, but alternatively it could mean being a lot more conservative about what you load for each request. Um, I want to be *less* conservative. Possibly *much* less. (like Gigabyes or even eventually Petabytes of shared data !) We appear to have drifted off the point. There's a big difference between data that an application needs to access and application variables. What you're describing is a database. If you want something more performant there are ways to optimise access to that amount of data, but if not I've completely lost what the problem is that you're trying to solve. Right now I have a need to store maybe 50Mb - 200Mb of data in RAM between requests. I suggesteed PetaBytes as an example of how much might be beneficial at some considerable point in the future to highlight how relatively un-scalable passing by copy (ie memcached / APC) is compared to application variables. This is essentially how ASP.NET Application variables and node.js work. Not a valid comparison. Node.js applications can only share variables within a single process, and they can do so because it's single-threaded. Once you scale your app beyond a single process you'd need to add a custom layer on to share data between them. I'm not sure about the architecture behind IIS and ASP.net but I imagine there are similar paradigms at work. I totally agree although, I *think* IIS uses multiple threads running in a single process (or Application Pool). I realise that ASP.NET / node.js have their own architectural issues but I'm confident that for enterprise applications (ie Drupal) the option for shared something is capable of many orders of magnitude higher performance and scalability than shared nothing. And that's why there are so many options around that enable such functionality. The need for something doesn't in any way imply that it should be part of the core system. Consider the impact such a requirement would have on the environment in which you run PHP. By delegating that feature to third-party modules, the PHP core doesn't need to concern itself with the details of how to share data between processes on every target platform. Agreed. If you were able to point me in the direction of such a 3rd party module I'd be a very happy man. APC and memcached are two of the most common examples, other than the vast array of DBMSes out there. Thanks, but APC And memcached are not even remotely comparable to Applications variables in terms of performance or memory efficiency. I'm surprised PHP doesn't already have Application variables, given that they are so similar to Session
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
On 31 Mar 2012, at 02:33, Simon wrote: Or: Why doesn't PHP have Applications variables like ASP.NET (and node.js) ? Hi, I'm working on optimising a php application (Drupal). The best optimisation I've found so far is to use APC to store various bits of Drupal data in RAM. The problem with this is that with Drupal requiring say 50Mb of data* per request is that lots of cpu cycles are wasted de-serialising data out of apc_fetch. Also 50Mb of data per http process !! is wasted by each one re-creating it's own copy of the shared data. 50MB? WTF is it storing?? I've never used Drupal, but based purely on that it sounds like an extremely inefficient piece of software that's best avoided! If it were possible for apc_fetch (or similar function) to return a pointer to the data rather than a copy of the data this would enable incredible reduction in cpu and memory usage. Vanilla PHP adheres to a principle known as shared nothing architecture in which, shockingly, nothing is shared between processes or requests. This is primarily for scalability reasons; if you stick to the shared nothing approach your application should be easily scalable. This is essentially how ASP.NET Application variables and node.js work. Not a valid comparison. Node.js applications can only share variables within a single process, and they can do so because it's single-threaded. Once you scale your app beyond a single process you'd need to add a custom layer on to share data between them. I'm not sure about the architecture behind IIS and ASP.net but I imagine there are similar paradigms at work. I'm surprised PHP doesn't already have Application variables, given that they are so similar to Session Variables and that it's been around for a long time in ASP / ASP.NET. Just because x does it, doesn't mean y should. I've used lots of languages over the years, including classic ASP, ASP.net, Perl, Python, Ruby, PHP (obv), and more, and I'm yet to see a compelling reason to want application variables. Let go of the possibility of application variables and your thinking will shift to other ways of solving the problem. I just wondered if there was a reason for not having this functionality or if it's on a road map somewhere or I've missed something :) ? As far as I am aware, ASP and ASP.net are the only web technologies to support application variables out of the box. You think that's simply because the others just haven't gotten around to it yet? -Stuart -- Stuart Dallas 3ft9 Ltd http://3ft9.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Could apc_fetch return a pointer to data in shared memory ?
On 31 Mar 2012, at 13:14, Simon wrote: Thanks again Stuart. On 31 March 2012 12:50, Stuart Dallas stu...@3ft9.com wrote: On 31 March 2012 11:19, Simon slg...@gmail.com wrote: Thanks for your answer. On 31 March 2012 09:50, Stuart Dallas stu...@3ft9.com wrote: On 31 Mar 2012, at 02:33, Simon wrote: Or: Why doesn't PHP have Applications variables like ASP.NET (and node.js) ? Hi, I'm working on optimising a php application (Drupal). The best optimisation I've found so far is to use APC to store various bits of Drupal data in RAM. The problem with this is that with Drupal requiring say 50Mb of data* per request is that lots of cpu cycles are wasted de-serialising data out of apc_fetch. Also 50Mb of data per http process !! is wasted by each one re-creating it's own copy of the shared data. 50MB? WTF is it storing?? I've never used Drupal, but based purely on that it sounds like an extremely inefficient piece of software that's best avoided! All sorts of stuff (taxonomies, lists of data, menu structures, configuration settings, content etc). Drupal is a sophisticated application. Besides, 50Mb of data seems like relatively tiny application state to want to access in fastest possible way. It's not hard to imagine wanting to use *much* more than this in future If it were possible for apc_fetch (or similar function) to return a pointer to the data rather than a copy of the data this would enable incredible reduction in cpu and memory usage. Vanilla PHP adheres to a principle known as shared nothing architecture in which, shockingly, nothing is shared between processes or requests. This is primarily for scalability reasons; if you stick to the shared nothing approach your application should be easily scalable. Yes, I know. I think the effect of this is that php will scale better (on average) in situations where requests don't need to share much data such as shared hosting. In an enterprise enviroment where the whole server might be dedicated to single application, shared nothing seems to be a synonym for re-load everything ? Yes, on one level that is what it means, but alternatively it could mean being a lot more conservative about what you load for each request. Um, I want to be *less* conservative. Possibly *much* less. (like Gigabyes or even eventually Petabytes of shared data !) We appear to have drifted off the point. There's a big difference between data that an application needs to access and application variables. What you're describing is a database. If you want something more performant there are ways to optimise access to that amount of data, but if not I've completely lost what the problem is that you're trying to solve. This is essentially how ASP.NET Application variables and node.js work. Not a valid comparison. Node.js applications can only share variables within a single process, and they can do so because it's single-threaded. Once you scale your app beyond a single process you'd need to add a custom layer on to share data between them. I'm not sure about the architecture behind IIS and ASP.net but I imagine there are similar paradigms at work. I totally agree although, I *think* IIS uses multiple threads running in a single process (or Application Pool). I realise that ASP.NET / node.js have their own architectural issues but I'm confident that for enterprise applications (ie Drupal) the option for shared something is capable of many orders of magnitude higher performance and scalability than shared nothing. And that's why there are so many options around that enable such functionality. The need for something doesn't in any way imply that it should be part of the core system. Consider the impact such a requirement would have on the environment in which you run PHP. By delegating that feature to third-party modules, the PHP core doesn't need to concern itself with the details of how to share data between processes on every target platform. Agreed. If you were able to point me in the direction of such a 3rd party module I'd be a very happy man. APC and memcached are two of the most common examples, other than the vast array of DBMSes out there. I'm surprised PHP doesn't already have Application variables, given that they are so similar to Session Variables and that it's been around for a long time in ASP / ASP.NET. Just because x does it, doesn't mean y should. I've used lots of languages over the years, including classic ASP, ASP.net, Perl, Python, Ruby, PHP (obv), and more, and I'm yet to see a compelling reason to want application variables. The reason that I'm suggesting this is because taking the example of Drupal, the ability to share information between requests by reference rather than by copy has the potential to be *millions* of times faster. Assuming I had say a 5Mb dataset that I wanted to re-use between