Experimental Design and Statistical Analysis Workshop

PORTLAND, OREGON

THURSDAY, 8 JULY. 14:00-18:00 		gecco logo
portland

Thomas Bartz-Beielstein,
Cologne University of Applied Science

Steffen Christensen,
Policy Research Initiative, Ottawa

Mike Preuss,
TU Dortmund

Mark Wineberg,
University of Guelph


Table of Contents

  1. Experimental Design and Statistical Analysis Workshop
    1. Table of Contents
    2. Overview
    3. Topics Covered
    4. Schedule
    5. Tasks and Hands-on Exercises
      1. T-1. Writing SPOT interfaces
      2. T-2. Performing experiments in the SPOT environment
    6. FAQs
      1. Q-1. How can R be installed on my local computer?
      2. Q-2. How can SPOT be installed on my local computer?
    7. Workshop Organizers
    8. References

Overview

This workshop is designed to be a hands-on companion to the Experimental Design and  Statistics tutorials:
 This workshop will give you a chance to try out the various techniques learned in the tutorials in a controlled environment so you could then apply them to your own problems when you get back to your lab. Our field is becoming more and more rigorous in the application of appropriate experimental design and statistical analysis for any experiment being submitted the major conferences in the field. Make sure that you or your students have the experience you need to be able to perform a proper analysis. You supply your laptop; we will supply the tools and techniques. Parts of this workshop use R, a freely available language and environment for statistical computing and graphics. R provides a wide variety of statistical and graphical techniques. The software can be downloaded from http://cran.r-project.org. 

The workshop will be presented at an introductory level and is meant for any EC researchers who want to compare their newly designed EC system with established EC systems to see if there is an improvement, or who want to determine which EC system performs better on their chosen problem; i.e., nearly everyone. It is vital, if our community is to be taken seriously, for us to continue to educate ourselves, and especially new Graduate students, on the statistical and experimental design techniques that are considered rigor for experiments performed in any field that is stochastic in nature, as is EC.

Topics Covered

Schedule

Tasks and Hands-on Exercises

T-1. Writing SPOT interfaces

How your algorithm can be integrated into the SPOT framework is explained in this tutorial.
Click here to go to the on-line version of this tutorial. The pdf can be downloaded here.

T-2. Performing experiments in the SPOT environment

This tutorial illustrates how experiments can be set up. It discusses problems related to experimental designs (factorial designs and space filling designs; regions of interest) and further settings that may influence experimental results.
Click here to go to the on-line version of this tutorial. The pdf can be downloaded here.

FAQs

Q-1. How can R be installed on my local computer?

Precompiled binary distributions of the base R system and contributed packages are available from the following resources:
 The R FAQ provides further help.

Q-2. How can SPOT be installed on my local computer?

You need to have a running R system. Follow the instructions from Q-1 first.
> install.packages("SPOT")
> library(SPOT)
> demo(spotDemo07RandomForestSann,ask=F)

Note, the option "ask=F" is necessary to avoid user interaction each time the output window is redrawn.

Workshop Organizers


bartz-beielstein
 Thomas Bartz-Beielstein, Cologne University of Applied Science.
Dr. Thomas Bartz-Beielstein is a professor for Applied Mathematics at Cologne University of Applied Sciences (CUAS). He is head of the FIWA research team at CUAS, which develops tools from Genetic and Evolutionary Computation (GEC) for applications from finance and water industry.
Prof. Bartz-Beielstein has published more than several dozen research papers, presented tutorials and workshops about interactive and automatic tuning  of randomized and deterministic algorithms, and has edited several books in the field of GEC, e.g., "Experimental Research in EC" and "Experimental Methods for the Analysis of Optimization Algorithms". His research interests include optimization, simulation, and statistical analysis of complex real-world problems.
Prof. Bartz-Beielstein is the driving force in the development of the Sequential Parameter Optimization methodology and the related Toolbox (SPOT). SPOT was applied as a tuner for numerous optimization algorithms such as evolution strategies, differential evolution, or particle swarm optimization.



Steffen Christensen, Policy Research Initiative, Ottawa.
Dr. Steffen Christensen is a Senior Policy Researcher at the Policy Research Initiative.
preuss
 Mike Preuss, TU Dortmund.
He is Research Associate at the Computer Science Department, University of Dortmund, Germany (since 2000), where he also received his Diploma degree in 1998. His research interests focus on the field of evolutionary algorithms for real-valued problems, namely on multimodal and multiobjective niching and the experimental methodology for (non-deterministic) optimization algorithms. He is currently working on the adaptability and applicability of computational intelligence techniques for various engineering domains and computer games.


wineberg
 Mark Wineberg, University of Guelph.
Dr. Mark Wineberg is Assistant Professor at the Department of Computing and Information Science, University of Guelph, Guelph, Ontario, Canada.


References