Kriging meta-modeling to verify traffic micro-simulation calibration methods

In the last decades simulation optimization has received considerable attention from both researchers and practitioners. Simulation optimization is the process of finding the best values of some decision variables for a system whose performance is evaluated using the output of a simulation model. A possible example of simulation optimization is the model calibration. In traffic modelling this topic is particularly relevant since the solutions to the methodological issues arising when setting up a calibration study cannot be posed independently. This calls for methodologies able to check the robustness of a calibration framework as well as further investigations of the issue, in order to identify possible “classes” of problems to be treated in a similar way. Therefore in the present work, first we describe a general method for verifying a traffic micro-simulation calibration procedure (suitable in general for simulation optimization), based on a test with synthetic data. Then we apply it to draw inference on the effect that different combinations of parameters to calibrate, optimization algorithm, measures of Goodness of Fit and noise in the data may have on the optimization problem. The time required to perform all the simulations needed to the study would make the present procedure unfeasible. For this reason a Kriging surrogate of the simulation model has been used in its place. Results show the importance of verifying the calibration procedure with synthetic data. In addition they ascertain the need for global optimization solutions, giving new insights into the topic.