If you're looking to integrate the tools into a larger application like,
say, a web-app, then I agree that piping has an ad-hoc feel to it.  We
usually just need to do experiments that don't involve end users, so piping
unix tools together is par for the course.  We also could have used sockets
or JNI if we wanted more direct inter-process communication.

I usually associate integration frameworks like the ones you mention as
being primarily concerned with integrating complex *data* across a large
number of applications with unique and intricate APIs.  I'm not clear on
how this will help you integrate ECJ with various simulators, since ECJ
will probably just treat your simulator as a single library call that
computes a single fitness value.

Perhaps you are more concerned about the problem of transferring statistics
information that ECJ collects on the EA's behavior back *out* into your
other application?


On Thu, Sep 3, 2015 at 12:50 AM, Adil Raja <[log in to unmask]> wrote:

> Dear Eric,
>    Thanks indeed for your kind reply. It has helped me develop clarity
> about how people use ECJ to integrate it with other software. I was really
> wondering how people really do such things. But I was thinking that this
> mechanism sounds a little ad-hoc. But its nice if it works. I am literally
> looking forward to this kind of dovetailing. But I want to do some serious
> dovetailing.
> What I have been doing since I sent this mail was that I looked up for
> Application Integration Frameworks that allow programmers to integrate
> different applications together (ECJ and MASON in this case, or some other
> simulator). What I found was a plethora of open source tools that can be
> used to achieve not only this, but also to orchesterate various parts of
> applications on cloud(s). One such tool is provided by and
> it is open source. The tool they provide is called AnyPoint platform. There
> are many other similar tools. A list can be found here:
> The reason why I have written the above paragraph is to get some
> reflection from you or other members of the list on the viability of such
> tools. I have not used AnyPoint yet but, among other things, it boasts to
> allow the user to integrate objects various objects and APIs from diffeernt
> applications together. It sounds pretty good. I was thinking that I might
> be able to use it to integrate ECJ with an external simulator in a nice
> way, and the implementation would also not be quite ad-hoc.
> I wonder what people have to say about this.
> Best Regards,
> Muhammad Adil
> On Wed, Sep 2, 2015 at 8:09 PM, Eric 'Siggy' Scott <[log in to unmask]>
> wrote:
>> Muhammad,
>> Everything except for #6 (the GUI) can be achieved by writing a special
>> objective function (i.e. by implementing the 'Problem' class in ECJ).
>> My group frequently uses ECJ to tune the parameters of models that have
>> been implemented in C++.  To use ECJ with an arbitrary external simulator,
>> what I've done is written a Problem that invokes an external process, and
>> pipes it a list of individuals on stdin.  The external program is then
>> responsible for decoding the genomes and returning either a fitness value,
>> or phenotypic measurements that the Problem can use to compute a fitness
>> value.
>> So, long story short, I'm not sure what specific additional "dovetailing"
>> work was done for MASON, but most of what you describe is simple to do with
>> ECJ and wouldn't require tinkering about with its internals more than usual.
>> Siggy
>> On Wed, Sep 2, 2015 at 3:50 AM, Adil Raja <[log in to unmask]> wrote:
>>> Dear all,
>>>    I want to use ECJ for a research project. I was wondering if anyone
>>> could provide me details about integrating ECJ with other software, written
>>> possibly in Java. I want to dovetail ECJ with another simulator that is
>>> written in Java. I saw on the website of ECJ that it has already been
>>> dovetailed with MASON, which has been cited as a sister project of ECJ, but
>>> actually it is a simulator of multi-agent systems. I want to integrate ECJ
>>> with other simulators such as the famous TORCS (The Open Racing Car
>>> Simulator). I was wondering if anyone has already tried such dovetailing of
>>> ECJ with other specific or arbitrary simulators. Moreover, I would want to
>>> know that what sort of expertise and tools would be required that would be
>>> most suitable for such an integration.
>>> Through the integration (dovetailing), I would want to be able to
>>> achieve certain objectives. These are listed as follows:
>>> 1) Through the integration, I should be able to instantiate an
>>> experiment in ECJ using one of the evolutionary computing algorithms that
>>> are already implemented in it.
>>> 2) The instantiated experiment should have individuals that are relevant
>>> to the agents in the simulation environment. This is to say that if the
>>> simulator is a car racing simulator, then I should be able to instantiate
>>> an experiment in ECJ, the members of which fully encapsulate the
>>> representation of the controllers of the cars. Same goes for any other type
>>> of simulator that simulates, for example, soccer players, or aircraft, or
>>> panels for solar energy systems.
>>> Perhaps I need to elaborate on this point a little bit more. Say that we
>>> have a car simulator that can have cars run in it. The cars are normally
>>> run by the means of their controllers (or microcontrollers). Suppose that I
>>> want to evolve controllers for the car(s) object(s) using ECJ. Now if I
>>> start an experiment in ECJ with a simple genetic algorithm, or genetic
>>> programming, or any variant of these, I should be able to specify the
>>> representation of the controllers to ECJ to encode as the genotype of the
>>> individuals of my experiment.
>>> I hope that I have explained this point well.
>>> 3) The dovetailing software should also be able to gather results and
>>> store them either in the ECJ experiment, to be used latter, or somewhere in
>>> itself from where it could be retrieved for analysis and reflection.
>>> 4) The dovetailing software should be general in nature. This is to say
>>> that it could achieve the above stated objectives for any of the
>>> evolutionary computing algorithms implemented in ECJ, and an arbitrary
>>> simulator, plenty of which are available online.
>>> 5) The software should also be able to save the states of the experiment
>>> and the simulator (if necessary) to be used at intermediate steps of
>>> experiments (evolution), and be able to pass the states back and forth
>>> between ECJ and the simulator if necessay.
>>> 6) The software should preferably have a nice GUI front-end so that
>>> anyone with minimal exposure to computer programming could invoke various
>>> types of experiments and play with them.
>>> 7) The dovetailing should also be able to leverage from the parallel
>>> processing abilities of ECJ and of the simulator, if possible.
>>> So this message is rather large. However, I shall be thankful to anyone
>>> who could provide me hints on as to how to implement such an application
>>> integration (dovetailing) software.
>>> I was also wondering that although it is mentioned on the website of ECJ
>>> that it is dovetailed with MASON, where could one find the implementation
>>> of that dovetailing.
>>> Thanks for reading this through,
>>> Looking forward to hearing from you,
>>> Best,
>>> Muhammad Adil
>> --
>> Ph.D student in Computer Science, George Mason University
>> CFO and Web Director, Journal of Mason Graduate Research


Ph.D student in Computer Science, George Mason University
CFO and Web Director, Journal of Mason Graduate Research