Open Access

Dennis Bystrow

• Annotated training data is essential for supervised learning methods. Human annotation is costly and laborsome especially if a dataset consists of hundreds of thousands of samples and annotators need to be hired. Crowdsourcing emerged as a solution that makes it easier to get access to large amounts of human annotators. Introducing paid external annotators however introduces malevolentAnnotated training data is essential for supervised learning methods. Human annotation is costly and laborsome especially if a dataset consists of hundreds of thousands of samples and annotators need to be hired. Crowdsourcing emerged as a solution that makes it easier to get access to large amounts of human annotators. Introducing paid external annotators however introduces malevolent annotations, both intentional and unintentional. Both forms of malevolent annotations have negative effects on further usage of the data and can be summarized as spam. This work explores different approaches to post-hoc detection of spamming users and which kinds of spam can be detected by them. A manual annotation checking process resulted in the creation of a small user spam dataset which is used in this thesis. Finally an outlook for future improvements of these approaches will be made.…