Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Le 23/11/2016 à 20:11, p...@highoctane.be a écrit : On Wed, Nov 23, 2016 at 4:16 PM, Thierry Goubier > wrote: 2016-11-23 15:46 GMT+01:00 p...@highoctane.be

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On Wed, Nov 23, 2016 at 4:16 PM, Thierry Goubier wrote: > > > 2016-11-23 15:46 GMT+01:00 p...@highoctane.be : > >> Thanks Thierry. >> >> Please also see that with new satellites, the resolution is ever >> increasing (e.g. Sentinel

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

2016-11-23 15:46 GMT+01:00 p...@highoctane.be : > Thanks Thierry. > > Please also see that with new satellites, the resolution is ever > increasing (e.g. Sentinel http://m.esa.int/Our_Activities/Observing_the_ > Earth/Copernicus/Overview4) > It has allways been so. Anytime

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Thanks Thierry. Please also see that with new satellites, the resolution is ever increasing (e.g. Sentinel http://m.esa.int/Our_Activities/Observing_the_Earth/Copernicus/Overview4) I understand the tile thing and indeed a lot of the algos work on tiles, but there are other ways to do this and

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi Phil, 2016-11-23 12:17 GMT+01:00 philippe.b...@highoctane.be < philippe.b...@gmail.com>: > [ ...] > > It is really important to have such features to avoid massive GC pauses. > > My use case is to load the data sets from here. > https://www.google.be/url?sa=t=web=j=http:// >

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Le 23 nov. 2016 12:07, "Igor Stasenko" a écrit : > > > > On 23 November 2016 at 12:41, p...@highoctane.be wrote: >> >> >> >> On Wed, Nov 23, 2016 at 10:51 AM, Igor Stasenko wrote: >>> >>> >>> >>> On 23 November 2016 at 10:50,

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 23 November 2016 at 12:41, p...@highoctane.be wrote: > > > On Wed, Nov 23, 2016 at 10:51 AM, Igor Stasenko > wrote: > >> >> >> On 23 November 2016 at 10:50, p...@highoctane.be >> wrote: >> >>> >>> >>> On Wed, Nov 23, 2016 at 12:53

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On Wed, Nov 23, 2016 at 10:51 AM, Igor Stasenko wrote: > > > On 23 November 2016 at 10:50, p...@highoctane.be > wrote: > >> >> >> On Wed, Nov 23, 2016 at 12:53 AM, Eliot Miranda >> wrote: >> >>> >>> >>> On Tue, Nov 22, 2016 at

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 23 November 2016 at 10:50, p...@highoctane.be wrote: > > > On Wed, Nov 23, 2016 at 12:53 AM, Eliot Miranda > wrote: > >> >> >> On Tue, Nov 22, 2016 at 10:26 AM, Sven Van Caekenberghe >> wrote: >> >>> >>> > On 22 Nov 2016, at 19:16,

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On Wed, Nov 23, 2016 at 12:53 AM, Eliot Miranda wrote: > > > On Tue, Nov 22, 2016 at 10:26 AM, Sven Van Caekenberghe > wrote: > >> >> > On 22 Nov 2016, at 19:16, p...@highoctane.be wrote: >> > >> > >> > >> > On Tue, Nov 22, 2016 at 5:57 PM, Igor Stasenko

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On Tue, Nov 22, 2016 at 10:26 AM, Sven Van Caekenberghe wrote: > > > On 22 Nov 2016, at 19:16, p...@highoctane.be wrote: > > > > > > > > On Tue, Nov 22, 2016 at 5:57 PM, Igor Stasenko > wrote: > > > > > > On 15 November 2016 at 02:18, Eliot Miranda

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On Tue, Nov 22, 2016 at 5:57 PM, Igor Stasenko wrote: > > > On 15 November 2016 at 02:18, Eliot Miranda > wrote: > >> Hi Phil, >> >> On Thu, Nov 10, 2016 at 2:19 AM, p...@highoctane.be >> wrote: >> >>> >>> >>> On Thu, Nov 10,

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 15 November 2016 at 02:18, Eliot Miranda wrote: > Hi Phil, > > On Thu, Nov 10, 2016 at 2:19 AM, p...@highoctane.be > wrote: > >> >> >> On Thu, Nov 10, 2016 at 10:31 AM, Denis Kudriashov >> wrote: >> >>> >>> 2016-11-10 9:49

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi Phil, On Thu, Nov 10, 2016 at 2:19 AM, p...@highoctane.be wrote: > > > On Thu, Nov 10, 2016 at 10:31 AM, Denis Kudriashov > wrote: > >> >> 2016-11-10 9:49 GMT+01:00 p...@highoctane.be : >> >>> Ah, but then it may be more

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On Thu, Nov 10, 2016 at 1:31 AM, Denis Kudriashov wrote: > > 2016-11-10 9:49 GMT+01:00 p...@highoctane.be : > >> Ah, but then it may be more interesting to have a data image (maybe a lot >> of these) and a front end image. >> >> Isn't Seamless something

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Le 14/11/2016 à 21:51, stepharo a écrit : Hi thierry did you happen to have a techreport or any description of your work? It's a chapter in my PhD thesis ;) Thierry Stef Le 11/11/16 à 11:44, Thierry Goubier a écrit : Le 11/11/2016 à 11:29, Stephan Eggermont a écrit : On 10/11/16

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi thierry did you happen to have a techreport or any description of your work? Stef Le 11/11/16 à 11:44, Thierry Goubier a écrit : Le 11/11/2016 à 11:29, Stephan Eggermont a écrit : On 10/11/16 21:35, Igor Stasenko wrote: No, no, no! This is simply not true. It is you, who writes the

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Le 11/11/2016 à 11:29, Stephan Eggermont a écrit : On 10/11/16 21:35, Igor Stasenko wrote: No, no, no! This is simply not true. It is you, who writes the code that generates a lot of statistical data/analysis data, and its output is fairly predictable.. else you are not collecting any data, but

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi, The discussion on this thread had nothing to do with FAMIX or Moose. It had to do with people’s ability to load larger pieces of data in the image without the VM imposing a low limit on it. There are clear scenarios where this is desirable, and I do not understand why this is a topic of

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 10/11/16 21:35, Igor Stasenko wrote: No, no, no! This is simply not true. It is you, who writes the code that generates a lot of statistical data/analysis data, and its output is fairly predictable.. else you are not collecting any data, but just a random noise, isn't? That would be green

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 10 November 2016 at 19:58, Stephan Eggermont wrote: > Igor wrote: > >Now i hope at the end of the day, > >the guys who doing data mining/statistical > >analysis will finally shut up and happily > >be able to work with more bloat without > >need of learning a ways to properly

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Igor wrote: >Now i hope at the end of the day, >the guys who doing data mining/statistical >analysis will finally shut up and happily >be able to work with more bloat without >need of learning a ways to properly >manage memory & resources, and >implement them finally. The actual problem is

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 10 November 2016 at 18:57, Tudor Girba wrote: > Hi Igor, > > I see you are still having fun :). I am not sure what you are arguing > about, but it does not seem to be much related to what I said. > > It is not fun, seeing that after years since we discussed this problem,

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 10 November 2016 at 18:41, Sven Van Caekenberghe wrote: > > > On 10 Nov 2016, at 17:35, Aliaksei Syrel wrote: > > > > > The speed of GC will always be in linear dependency from the size of > governed memory. > > > > Asymptotic complexity of GC is O(N),

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 10 November 2016 at 17:41, Sven Van Caekenberghe wrote: > Even that is not necessarily true, Generational Garbage collection and > other tricks can avoid a full heap GC for a long time, even (or especially) > under memory allocation stress. That is why it is Big O notation

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi Igor, I see you are still having fun :). I am not sure what you are arguing about, but it does not seem to be much related to what I said. And again, I would be very happy to work with you on something concrete. Just let me know if this is of interest and perhaps we can channel the energy

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

> On 10 Nov 2016, at 17:35, Aliaksei Syrel wrote: > > > The speed of GC will always be in linear dependency from the size of > > governed memory. > > Asymptotic complexity of GC is O(N), where N is heap size - amount of > objects, not memory size. Even that is not

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

> The speed of GC will always be in linear dependency from the size of governed memory. Asymptotic complexity of GC is O(N), where N is heap size - amount of objects, not memory size. I agree, however, that it's not good to create a lot of short living objects. That is why there are many

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 10 November 2016 at 11:42, Tudor Girba wrote: > Hi Igor, > > I am happy to see you getting active again. The next step is to commit > code at the rate you reply emails. I’d be even happier :). > > To address your point, of course it certainly would be great to have more

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

2016-11-10 12:38 GMT+01:00 Norbert Hartl : > > Am 10.11.2016 um 12:27 schrieb Thierry Goubier >: > > > > 2016-11-10 12:18 GMT+01:00 Norbert Hartl : > >> [ ...] >> >> Be it small chunks of data or not. A statement that general is

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

> Am 10.11.2016 um 12:27 schrieb Thierry Goubier : > > > > 2016-11-10 12:18 GMT+01:00 Norbert Hartl >: > [ ...] > > Be it small chunks of data or not. A statement that general is most likely to > be wrong. So the

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

2016-11-10 12:18 GMT+01:00 Norbert Hartl : > [ ...] > > Be it small chunks of data or not. A statement that general is most likely > to be wrong. So the best way might be to ignore it. Indeed you are right > that hardware got cheap. Even more important is the fact that

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

> Am 10.11.2016 um 10:42 schrieb Tudor Girba : > > Hi Igor, > > I am happy to see you getting active again. The next step is to commit code > at the rate you reply emails. I’d be even happier :). > +1 > To address your point, of course it certainly would be great to

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On Thu, Nov 10, 2016 at 11:43 AM Tudor Girba wrote: > Hi Igor, > > I am happy to see you getting active again. The next step is to commit > code at the rate you reply emails. I’d be even happier :). > > > aouch that was not very nice I agree with Igor and Phil, there

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On Thu, Nov 10, 2016 at 10:31 AM, Denis Kudriashov wrote: > > 2016-11-10 9:49 GMT+01:00 p...@highoctane.be : > >> Ah, but then it may be more interesting to have a data image (maybe a lot >> of these) and a front end image. >> >> Isn't Seamless something

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

> On 10 Nov 2016, at 10:25, Sven Van Caekenberghe wrote: > > >> On 10 Nov 2016, at 10:10, Denis Kudriashov wrote: >> >> >> 2016-11-09 23:30 GMT+01:00 Nicolas Cellier >> : >> uptime 0h0m0s >> memory

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi Igor, I am happy to see you getting active again. The next step is to commit code at the rate you reply emails. I’d be even happier :). To address your point, of course it certainly would be great to have more people work on automated support for swapping data in and out of the image. That

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

2016-11-10 9:49 GMT+01:00 p...@highoctane.be : > Ah, but then it may be more interesting to have a data image (maybe a lot > of these) and a front end image. > > Isn't Seamless something that could help us here? No need to bring the > data back, just manipulate it through

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi, There is never any point in talking down on people. It never leads to anything except perhaps stifling action and participation. We want to foster an environment in which people should not be afraid to be a novice at something and ask for help. Cheers, Doru > On Nov 10, 2016, at 9:45 AM,

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

> On 10 Nov 2016, at 10:10, Denis Kudriashov wrote: > > > 2016-11-09 23:30 GMT+01:00 Nicolas Cellier > : > uptime 0h0m0s > memory 70,918,144 bytes > old 61,966,112 bytes

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi Igor. 2016-11-10 9:12 GMT+01:00 Igor Stasenko : > Because even if you can fit all data in memory, consider how much time it > takes for GC to scan 4+ Gb of memory, comparing to > 100 MB or less. > But do you think there is no solution to that. Imaging Pharo as computer

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

2016-11-09 23:30 GMT+01:00 Nicolas Cellier < nicolas.cellier.aka.n...@gmail.com>: > uptime 0h0m0s >> memory 70,918,144 bytes >> old 61,966,112 bytes (87.4%) >> young 2,781,608 bytes (3.9004%) >> > I see

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Ah, but then it may be more interesting to have a data image (maybe a lot of these) and a front end image. Isn't Seamless something that could help us here? No need to bring the data back, just manipulate it through proxies. FWIW, I have 2PB of data. Not going to fit in RAM. But also would takes

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Tudor, Igor still has a point here. I was talking yesterday with a data science guy and he was indeed more interested in lamenting than working out solutions for his problems. Which weren't that hard to begin with as all it took is an hour of work to get his results. But I think he felt better

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

On 10 November 2016 at 07:27, Tudor Girba wrote: > Hi Igor, > > Please refrain from speaking down on people. > > Hi, Doru! I just wanted to hear you :) > If you have a concrete solution for how to do things, please feel free to > share it with us. We would be happy to

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

> Am 10.11.2016 um 07:27 schrieb Tudor Girba : > > Hi Igor, > > Please refrain from speaking down on people. +1 > If you have a concrete solution for how to do things, please feel free to > share it with us. We would be happy to learn from it. > +10 Norbert > Cheers,

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Hi Igor, Please refrain from speaking down on people. If you have a concrete solution for how to do things, please feel free to share it with us. We would be happy to learn from it. Cheers, Tudor > On Nov 10, 2016, at 4:11 AM, Igor Stasenko wrote: > > Nice progress,

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Ok, I'll try that in the coming days on my desktop box. I've got 32GB on that box, so it will be interesting. What would be a good way to stress test the GC? Phil On Wed, Nov 9, 2016 at 3:59 PM, Thierry Goubier wrote: > > > 2016-11-09 15:23 GMT+01:00

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

2016-11-09 15:23 GMT+01:00 philippe.b...@highoctane.be < philippe.b...@gmail.com>: > Anyone having tested such 64 bit goodness on a Linux box? > Only as a professional demo for code developped up to the very last moment on Pharo 32 bits. So I didn't try to explose the available RAM on my laptop

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

> > > OK, I am quite excited about the future possibilities of 64-bit Pharo. So > I played a bit more with the current test version [1], trying to push the > limits. In the past, it was only possible to safely allocate about 1.5GB of > memory even though a 32-bit process' limit is theoretically

Re: [Pharo-dev] Breaking the 4GB barrier with Pharo 6 64-bit

Very nice! Thanks for the heads up Sven. > On 9 Nov 2016, at 12:06, Sven Van Caekenberghe wrote: > > OK, I am quite excited about the future possibilities of 64-bit Pharo. So I > played a bit more with the current test version [1], trying to push the > limits. In the past, it