ULMFIT by Howard and Ruder holds several state-of-the-art on text classification. The technique comprises several steps of transfer learning with language models. Language models try to predict the next words based on previous words. They can trained unsupervised on large collection of texts. As the by-product, they create powerful context-aware text represenations. While working on my master’s thesis, I needed a German language model. I created one and publish it for further usage.
The model and an exemplary notebook are on GitHub: https://github.com/jfilter/ulmfit-for-german