Hey again,
I've found the bottleneck of my algorithm. Drools was running in infinite loops, using only one core and letting the others wait.
So I've solved this problem by putting a limit how many rules it shall fire max. But fixing this bug, revealed a more suspicious bug.
Sometimes ECJ runs with all threads waiting. I've checked if the problem comes from Drools again, but this time it's after/before
Drools run. I've checked which threads are running with VisualVM, getting the result that only Attach Listener, Signal Dispatcher
and some RMI TCP threads are running, while all my threads 0-14 are waiting.
So what would I'd like to know is, what could cause all threads to wait? Doesn't the merger handle the threads after they're finished?
By the way it seems that this bug occurs after Drools has finished and the rule has been evaluated, but I'm not sure because I don't
know how many times it needs to run. (Depends on Generations and Population it needs)
Greetings
Bojan
----- Ursprüngliche Mail -----
Von: "Bojan Janisch" <[log in to unmask]>
An: "ECJ Evolutionary Computation Toolkit" <[log in to unmask]>
Gesendet: Mittwoch, 19. Juni 2013 14:32:40
Betreff: Re: ECJ is not using all cores
Hi Uday,
I've specified the cores if you look at my previous answer to Warren. Are there some more
places where to define multi-threading for ECJ? I don't find any other information than
setting the cores with breedthreads and evalthreads.
Greetings
Bojan
----- Ursprüngliche Mail -----
Von: "Uday kamath" <[log in to unmask]>
An: [log in to unmask]
Gesendet: Mittwoch, 19. Juni 2013 13:30:39
Betreff: Re: ECJ is not using all cores
Bojan
ECJ can be multi threaded, look at the manual, various elements of
breeding, evaluating, etc can be multithreaded and you can specify
threads=cores to get full parallelism. I am running ECJ on 12 core and
using all of them, so shouldn't be an issue.
Uday
On Wed, Jun 19, 2013 at 5:13 AM, Bojan Janisch
<[log in to unmask]> wrote:
> Hello everyone,
>
> I'm using ECJ for rule-creation and -evaluation. Recently I can use ECJ on 20 cores
> of a server which runs very fast at the beginning. But after a while the CPU-usage
> goes down to 1 core which I don't understand.
>
> The evaluation process looks so:
>
> Step 1: Create a valid rule from GPTree
> Step 2: If rule contains nothing useful skip everything else, set bad score and go to Step 6
> Step 3: Rule contains something useful, so load the testdata
> Step 4: Run the rule through Drools and testdata (Drools is a ruleengine which also runs multithreaded)
> Step 5: Compare the output of Drools and a goldstandard (the goldstandard and some more information are one time loaded at setup)
> Step 6: Put the score, returned by the comparison, into the koza fitness
>
> I don't know why but after some time, ECJ is only using 1 core.
> Does someone got an idea where the problem could come from?
>
> Greetings
>
> Bojan
|
