Robert Baruch wrote:

 > When I wasn't using Master/Slave, I could just put the calculation in my
 > Problem's finishEvaluating method.
 >
 > However, with Master/Slave, my Problem's finishEvaluating method isn't
 > called.


Got it.  MasterProblem.finishEvaluating is getting called but doesn't 
call the underlying Problem method.  So some explanation.

At present ECJ's generational Problem forms can expect more or less 
three basic forms of batched individual evaluation:

1.  Each individual is evaluated and his fitness is assessed 
immediately.  This is the "old" form.  In this case, the Problem is 
called like this:

	evaluate()
	evaluate()
	...

2. Each individual's evaluate() method is called, but their fitnesss 
don't have to be set immediately.  This allows the underlying Problem to 
batch-evaluate them (by shipping them off to a remote slave for 
example).  In this case, the Problem is called like this:

	prepareToEvaluate()
	evaluate()
	evaluate()
	...
	finishEvaluating()  // synchronizes fitness-setting

3. Each individual's evaluate() method is called, but there's no 
guarantee at all when they'll be ready, and no synchronization.  When an 
individual is ready, it's ready.  For example, asynchronous steady-state 
evolution works like this.  In this case, the Problem is called like this:

	prepareToEvaluate()
	if (canEvaluate())
		evaluate()
	if ((ind = getNextEvaluatedIndividual()) != null)
		ind's evaluation is done, so
		introduce ind to population
	if (canEvaluate())
		evaluate()
	if ((ind = getNextEvaluatedIndividual()) != null)
		ind's evaluation is done, so
		introduce ind to population
	...

The reason for the various scenarios is because of master/slave 
situations.  In those situations, ECJ replaces your Problem instance 
with a special instance called a MasterProblem.  This thing pretends to 
be your Problem, but in fact when it receives individuals to evaluate, 
it ships them off to remote slaves.  There on the slave planets, evil 
alien taskmasters fire up *real* instances of *your* Problem subclass to 
test the individuals.  When their fitnesses are assessed, the 
information is relayed back to the home planet, where the MasterProblem 
then reports appropriately.

What this means is that, more or less, your Problem is never actually 
used on the master.  It's only fired up on the slave. MasterProblem 
holds a copy of the Problem (in its 'problem' instance variable) but it 
never calls it or uses it!

So what can we do?  Well, one thing we could do is hack 
MasterProblem.finishEvaluating() and MasterProblem.prepareToEvaluate() 
to call problem.finishEvaluating() and problem.prepareToEvaluate() at 
the end of MasterProblem's versions of the methods.  That should work 
for you for the moment, assuming you're doing straight generational 
master/slave evaluation, you could modify MasterProblem.java like this:


// prepare for a batch of evaluations
public void prepareToEvaluate(final EvolutionState state, final int 
threadnum)
   {
   if (jobSize > 1) queue = new ArrayList();
   batchMode = true;
   problem.prepareToEvaluate(state, threadnum);
   }


// wait until a batch of evaluations is finished
public void finishEvaluating(final EvolutionState state, final int 
threadnum)
   {
   if(showDebugInfo)
     state.output.message(Thread.currentThread().getName() + "Waiting 
for all slaves to finish.");
   flush(state, threadnum);
   queue = null;  // get rid of it just in case

   monitor.waitForAllSlavesToFinishEvaluating( state );
   batchMode = false;
   if(showDebugInfo)
     state.output.message(Thread.currentThread().getName() + "All slaves 
have finished their jobs.");
   problem.finishEvaluating(state, threadnum);
   }

Understand that your Problem (on the master) will never see an 
evaluate() method called.  Only your Problems on the slaves will have 
seen one.  You'll just get a prepareToEvaluate() and then a 
finishEvaluating() and that's it.  On the slaves, on the other hand, 
your Problem will only have evaluate() called on it!  As long as you're 
not gathering statistics in evaluate() on the side that you need to use 
in finishEvaluating(), this shouldn't be a problem.

I have to think about whether or not doing this hack permanently is a 
good idea or not.  It may not be.  But it should work for you.

The other thing you could do is tap in somewhere else: for example, you 
could create a Statistics class which overrides postEvaluationStatistics 
to do the modifications you like before breeding and selection occurs. 
That'd probably work too.

Sean