Thanks for your fast and valuable comments. I increased the Artificial
Ant timestep to 600, as you suggested, and it seems to work :). It might
be of interest to someone in the list the results of my experiments, so
I left them here.
Koza I (p202): Imin=450k, SR=7%
ECJ: Imin=666K, SR=11%
Notes: 100 runs, M=500, timestep set to 600
Koza I (p196): Imin=245k, SR=75%
ECJ: Imin=427k, SR=70%
Notes: 50 runs, M=500, G=200, fitness proportionale selection used
instead of tournament
Koza I (p203): Imin=162k, SR=34%
ECJ: Imin=188k, SR=30%
Notes: 100 runs, M=500, ECJ default parameters
Koza I (p543): SR=0%
ECJ: Imin=3.38E7, SR=4%
Notes: 50 runs, M=4000
EVEN-5 PARITY WITH ADF
Koza I (p544): Imin=152k, SR=5%
Notes: 100 runs, M=4000
Sean Luke escribió:
> There is always the definite possibility of a bug in ECJ: for example, I
> just routed out a long-standing one in Lawnmower recently (check CVS).
> But another reason may be that Koza-I and Koza-II'ss results have proven
> historically difficult to reproduce. There are a number of reasons for
> this, but here are two: the random number generator used in Koza-I and
> Koza-II was troublingly non-random; and certain variables are not as
> you'd expect in print. For example, though Koza-I says 400 timesteps
> for Artificial Ant, in fact it's believed he used 600. So much
> subsequenty work used 400 timesteps as a result that that's the default
> for both ECJ and lil-gp. You an change it to 600 and see what's up.
> There are other explanations for differences which have been offered
> which I don't want to get into here.
> So: if we can nail down a bug, I'd love to find out. But be prepared
> for the probability that it's Koza-I and not ECJ. One way to check for
> the possibility of an ECJ bug in certain problems is to run lil-gp and
> see which one it more or less agrees with. Let me know what you get.
> On Oct 27, 2009, at 5:37 PM, David F. Barrero wrote:
>> Dear all,
>> I am trying to reproduce Koza's performance curves for some typical GP
>> problems implemented in ECJ, but I had a limited success. Some problems
>> such as simple regression have similar performance curves compared with
>> the ones shown in Koza I, while others (for instance, Santa Fe trail or
>> even 5 parity with ADFs) yields very different curves.
>> In theory, should ECJ examples generate the same performance curves than
>> those shown in Koza I/II?.
>> Thanks in advance,
David F. Barrero
Departamento de Automática
Universidad de Alcalá
Tlf: (34) 91-885-69-20
Web : http://atc1.aut.uah.es/~david/
Jabber/GTalk: [log in to unmask]
Departamento de Automática
Ctra. Madrid-Barcelona km 31,600
28871 Alcalá de Henares. Madrid