Please use this identifier to cite or link to this item:
Appears in Collections:Computing Science and Mathematics Conference Papers and Proceedings
Peer Review Status: Refereed
Author(s): Woodward, John
Brownlee, Alexander
Johnson, Colin
Contact Email:
Title: Evals is not enough: why we should report wall-clock time
Editor(s): Friedrich, T
Citation: Woodward J, Brownlee A & Johnson C (2016) Evals is not enough: why we should report wall-clock time In: Friedrich T (ed.) GECCO '16 Companion Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion. GECCO 2016: Genetic and Evolutionary Computation Conference, New York, 20.07.2016-24.07.2016. New York: ACM, pp. 1157-1158.
Issue Date: 2016
Conference Name: GECCO 2016: Genetic and Evolutionary Computation Conference
Conference Dates: 2016-07-20 - 2016-07-24
Conference Location: Denver, CO, USA
Abstract: Have you ever noticed that your car never achieves the fuel economy claimed by the manufacturer? Does this seem unfair? Unscientific? Would you like the same situation to occur in Genetic Improvement? Comparison will always be difficult [9], however, guidelines have been discussed [3, 5, 4]. With two GP [8] approaches, comparing the number of evaluations of the fitness function is reasonably fair. This means you are comparing the GP systems, and not how well they are implemented, how fast the language is. However, the situation with GI [6, 1] is unique. With GI we will typically compare systems which are applied to the same application written in the same language (i.e. a GI systems targeted at Java, may not even be applied to C). Thus, wall-clock time becomes more relevant. Thus, this paper asks if reporting number of evaluations is enough, or if wall-clock time is also important, particularly in the context of GI. It argues that reporting time is even more important when doing GI when compared to traditional GP.
Status: AM - Accepted Manuscript
Rights: Publisher policy allows this work to be made available in this repository. Published in 10.1145/2908961.2931695. The original publication is available at:

Files in This Item:
File Description SizeFormat 
giTime07 (1).pdfFulltext - Accepted Version158.42 kBAdobe PDFView/Open

This item is protected by original copyright

Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

If you believe that any material held in STORRE infringes copyright, please contact providing details and we will remove the Work from public display in STORRE and investigate your claim.