Hmm... maybe it would be even easier to just let slaves connect to
the master whenever they want, and accept M jobs. That is, the master
doesn't assign jobs to slaves a priori, and would therefore only have
to keep track of slaves with outstanding jobs. This way, slaves that
return quickly will automatically get assigned more jobs?
On Oct 26, 2006, at 2:36 PM, Sean Luke wrote:
> If the documentation says something along those lines, we need to
> change it. Actually the way the mechanism works is somewhat
> When a slave comes online, it joins a pool of available slaves.
> When the system must evaluate N individuals, say, a population's
> worth, it goes through the following loop:
> while there are still individuals to be processed
> pick an available slave arbitrarily
> give that slave some (small number) M individuals
> In the background, it waits for slaves to come back finished, marks
> those individuals as finished, and the slave enters the available
> list again. If a slave goes down while processing, its unprocessed
> individuals are put back in the need-to-be-processed list again.
> It's true ECJ will assign jobs to every one of your processors if
> there are many more individuals than processors. But if you have a
> fast processor on one machine and a slow one on another, the fast
> processor will come available more often and balance the load.
> The advantage of this mechanism over your proposal (which is good
> btw) is that slow processing isn't just a function of the CPU
> speed. It's more often than not a function of the size and
> complexity of the individual. And that's not something you can
> necessarily figure out a priori beforehand.
> Still there might be some improvements, mostly in the
> identification of which slave to pick. If a slave has a record of
> coming back rapidly, maybe it ought to go to the front of the line
> per se.