In English

Predicting annotation difficulty using Monte Carlo dropout

Jens Wilhelmsson
Göteborg : Chalmers tekniska högskola, 2019. 55 s.
[Examensarbete på avancerad nivå]

Developing products based on machine learning algorithms require relevant and accurate datasets. In particular when it comes to supervised learning algorithms whose performance is directly related to the quality and amount of training data. Within the field of computer vision, classification is a task that require training data in the form of annotated images. Annotating images is a manual task and I propose that the annotation difficulty of an image should be interpreted as the likelihood of someone else annotating an image differently. Knowing in advance which images are hard to annotate would facilitate the distribution of work between annotators with varying experience. In this thesis, it is shown that the uncertainty derived from Monte Carlo dropout resembles the variance of a group of persons annotations of the same image. This finding indicates that the level of agreement between persons can be predicted, and thus enable for better distribution of work between annotators. Furthermore, the finding could also be used to order images during training by prioritizing harder images higher.

Nyckelord: Annotations, Deep learning, Convolutional neural networks, Annotation difficulty, Monte Carlo dropout

Publikationen registrerades 2019-06-12. Den ändrades senast 2019-06-12

CPL ID: 256747

Detta är en tjänst från Chalmers bibliotek