On Oct 11, 2013, at 23:47, Sean Luke <[log in to unmask]> wrote:
> On Oct 11, 2013, at 4:55 PM, Ralf Buschermöhle wrote:
>> in order to make it even more complex ... also different multithreaded sized clients would be ... great! :)
> That one would be too tough to implement anytime soon.
> But as to waiting until the job has all come in: my analysis of the current code suggests that this is NOT what happens.
> Slaves can run in one of two modes:
> 1. "run-evolve mode". Here the slave loads the job into a population, then evaluates them, and if there's more time, does some evolution on that population, eventually returning the revised population as new versions of the original individuals.
The different modes are triggered by ...
# return complete individuals
eval.return-inds = true
> 2. NOT "run-evolve mode" (the default). Here the slave fires off a new thread each time it reads an individual in, and immediately starts evaluating it. Then it returns all the fitness results together as a group.
> The downside of this is that if your job size is much larger than your threads, you're doing too much context switching. I can fix that. Also it waits until everything's finished before the fitnesses are returned; this probably isn't that big of an issue in truth. But the upside is that it DOES immediately start processing an individual as soon as one arrives -- it doesn't wait until the job's been loaded.
// just to make sure I understand (some) implications correctly ...
Let's assume there is a multithreaded slave with 4 cores with eval.masterproblem.job-size = 1 and eval.masterproblem.max-jobs-per-slave = 8 with a total population of some hundred individuals ...
0. The server fills the queue of the client (completely) with 8 jobs
1. The slave would evaluate 4 jobs concurrently
2. After finishing the first job (the others take significantly longer) the slave sends the result to the server and starts processing the 5. job in the queue
3. The server would fill up the job queue of the slave after receiving the evaluated individual (if eval.masterproblem.job-size = 2 the server would wait until 2 individuals have been received and then refills the queue) as long as individuals need to be processed.
Meaning that eval.masterproblem.max-jobs-per-slave defines the maximal concurrency level for each client and eval.masterproblem.job-size defines the "chunk" size of communication fragments between client and server.
Is the scenario correct? Actually this would be fantastic! :)