Print

Print


On Jan 29, 2006, at 10:07 AM, Dignum, Stephen A N wrote:

> I'm working through some of the GP examples in the apps directory.  
> All work fine until I start to up the population size (doubling to  
> 2048). I then get an out-of-memory error.

Actually, we use 2048 all the time and don't have much trouble, so  
that's interesting to hear.  Which examples are creating difficulty  
for you?

> To get around this I increase the standard java memory parameters  
> (Xmx, Xms, XX: etc), This works but if I then increase the number  
> of generations past about 65 I get a sudden reduction in  
> performance which isn't due to increase in program size i.e. number  
> of nodes stays about the same. There may be a large increase in  
> program depth but from my understanding of how ECJ works (by  
> looking through the code!) it is the number of nodes that effects  
> performance not depth.

ECJ's memory usage is fairly constant.  In addition to a few small  
constant elements, just after breeding time (the worst memory usage  
case) ECJ has the current population _and_ the previous population  
which was generated from it.

In the GP examples, individuals in the population have a constant  
overhead each: the big loss comes from GPNodes, which take up (last  
time I checked) about 32 bytes apiece.  Compared to other methods,  
it's a memory hog.  I got into significant discussion of this, I  
think, in the last tutorial.

So as the individuals grow in size due to bloat, it's perfectly  
plausible for ECJ to be growing beyond your standard heap size.  But  
a sudden reduction in performance shouldn't ever happen if it's not  
commensurate with a sudden increase in GP nodes.  There is one  
possibility though: you could be out of your sweet spot and are now  
swapping badly.  Could this be problems specific to Windows?  We  
typically run on Linux and OS X.

You can see the current memory usage and swap information by running  
like this

	java -verbose:gc ...

Sean