Synthetic data to develop a trustworthy autonomous driving system | Chapter 6

Synthetic data to develop a trusthworthy autonomous driving system


Share on linkedin
Share on twitter


Hamid Serry, WMG Graduate Trainee, University of Warwick

Last week we looked at Spatial Recall Index, a very new performance metric, and how it could be used within the scope of training and assessing the performance of object detection neural network models. This week, we are assessing a different class of variable: Hyper Parameters, and more specifically Hyper Parameter Tuning.


Hyperparameters are fixed variables which are used, within machine learning, to control the training process and determine the quality and efficiency of the resulting network. They are dictated before training has begun, so scheduled values such as changes to learning rate over the epochs could be set in advance, and cannot be manually changed during the training, so selecting effective parameters is an important task.

Our main focus are the parameters relating to the training algorithm, a sample of which is as follows:

Hyperparameter Tuning

Hyper parameter tuning fully trains a model under specific hyper parameters, and then repeats this process, varying the hyperparameter values in order to optimise performance of the network with relation to a main performance metric (such as mAP in this case). This tracks the performance of each hyperparameter combination and is used to select the final configuration of hyperparameters with the best mAP.

Effective tuning processes will remove parameters which have little to no effect on the overall performance of the model, as well as end a cycle early when it is clearly less successful than other trained models. At the end of the tuning, the best parameters can be selected for the model deployment. There are many methods such as manual tuning [3] and other complicated methods such as Bayesian Optimisation [4].

As this process is equivalent to repeating a training procedure many times, in order to find the most efficient solution, it requires a much larger processing time than simply one training cycle. Although this process can be a very expensive endeavour, the results can show a very significant difference between distinct hyperparameter setups, leading to some organisations keeping their parameters
secret, and opened the doors for hyper parameter theft [5].


We have discussed some of the various hyperparameters used within machine learning, and how they can be altered to increase performance of a model. Implementing the optimisation to the current selected neural network will allow us to optimise performance, and achieve a more precise model to test the generated Anyverse dataset on.


[2] G. Zhang, C. Wang, B. Xu and R. Grosse, “Three Mechanisms of Weight Decay Regularization”, 2018. arXiv preprint arXiv:1810.12281.

[5] B. Wang and N. Z. Gong, “Stealing Hyperparameters in Machine Learning,” 2018 IEEE Symposium on Security and Privacy (SP), 2018, pp. 36-52,

Read more >>>

Looking to start your Synthetic Data journey or need help with your current project? We'd love to know more.

Looking for the right Synthetic Data to speed up your system? Please, enter the Anyverse now

Let's talk about synthetic data!

[contact-form-7 404 "Not Found"]