I also had problems trying to debug my ABM code. The technique I
found most useful was to fix the random seed, which makes MASON run
deterministically. I could then identify the bug, eliminate it, and
then try with other random seeds to see if there are other bugs as well.
On Aug 25, 2006, at 6:15 PM, Tony Bigbee wrote:
> Liviu Panait wrote:
>> I believe one of the problems with deterministic scheduling is
>> that you may start seeing all sorts of artifacts.
> You are right, but Mike's point is still an interesting one.
> Having a means to verify correct code functioning is of very high
> value and running experiments n times over to smooth out single run
> outcomes can be time-consuming. And there is a paucity of easy to
> use 'tools' to help understand the behavior produced in emergent
> sims. When I tried to replicate Growing Artificial Societies
> (Epstein and Axtell's Sugarscape) results, I found that I had many
> kinds of bugs that were hard to track down and that my incorrect
> implementations may be partially or fully responsible for why I
> didn't fully replicate the simulation outcomes documented in the book.
> For what it's worth, one leading ABM practicioner claimed that he
> had done experiments to demonstrate that randomizing only 20% of a
> schedule was sufficient to eliminate artifacts. I don't think this
> has been published.
> But you would have to turn off other random elements as well, not
> just the scheduler, to make the simulation deterministic enough,
> depending on the debugging or verification goal. From an
> unpublished paper on my effort to replicate the simulation outcomes
> described in GAS, here is an attempt at a randomization typology
> for agent-based models:
> Type 1 - Initial distributions of entities and their states, such
> as agents' genetic characteristics and initial locations.
> Type 2 - Dynamic state assignments from distributions such as the
> next direction to look in agent visual scan patterns or an agent
> life expectancy at birth.
> Type 3 - Order of execution among entities (such as agents), and
> between agents for asynchronous interactions. In both the classic
> and MASON models, agents are randomly selected for execution/
> behavior each time step, but the sequence of agent and environment
> rules execution is deterministic.
> Type 4 - Simulation outcomes that can be characterized by
> theoretical tools such as Markov fields. For example, Culture (K)
> eventually reaches equilibrium in one of three absorbing states
> (all blue, static levels of the two colors on the sugar mountains,
> or all red)
> A search of the MASON Sugarscape source code indicates that there
> are 24 statements -- calls to random.next()) -- for Type 1 and
> Type 2 randomization. I implemented about 75% of the rules in GAS.