1

Ungoliant: An Optimized Pipeline for the Generation of a Very Large-Scale Multilingual Web Corpus

We propose a new pipeline that is faster, modular, parameterizable, and well documented. We use it to create a corpus similar to OSCAR but larger and based on recent data.

Julien Abadji, Pedro Ortiz Suarez, Laurent Romary, Benoît Sagot

Ungoliant: An Optimized Pipeline for the Generation of a Very Large-Scale Multilingual Web Corpus

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

We audit 5 multilingual corpora, finding that lower-resource corpora have systematic issues.

Isaac Caswell, Julia Kreutzer, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Rubungo Andre Niyongabo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, Mofetoluwa Adeyemi

SinNer@CLEF-HIPE2020: Sinful Adaptation of SotA models for Named Entity Recognition in Historical French and German Newspapers

In this article we present the approaches developed by the Sorbonne-INRIA for NER (SinNer) team for the CLEF-HIPE 2020 challenge on Named Entity Processing on old newspapers.

Pedro Ortiz Suarez, Yoann Dupont, Gaël Lejeune, Tian Tian

A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages

We explore the impact of the training corpus on contextualized word embeddings in five mid-resource languages.

Pedro Ortiz Suarez, Laurent Romary, Benoît Sagot

A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages

CamemBERT: a Tasty French Language Model

We explore the impact of the training data size on a French version of RoBERTa. (Equal contribution by the first three authors).

Louis Martin, Benjamin Muller, Pedro Ortiz Suarez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, Benoît Sagot

CamemBERT: a Tasty French Language Model

Building a User-Generated Content North-African Arabizi Treebank: Tackling Hell

We introduce the first treebank for a romanized user-generated content variety of Algerian, a North-African Arabic dialect.

Djamé Seddah, Farah Essaidi, Amal Fethi, Matthieu Futeral, Benjamin Muller, Pedro Ortiz Suarez, Benoît Sagot, Abhishek Srivastava

Les modèles de langue contextuels Camembert pour le Français : impact de la taille et de l'hétérogénéité des données d'entrainement

We explore the impact of the training data size and heterogeneity on French language modeling. (Equal contribution by the first three authors).

Louis Martin, Benjamin Muller, Pedro Ortiz Suarez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Benoît Sagot, Djamé Seddah

Les modèles de langue contextuels Camembert pour le Français : impact de la taille et de l'hétérogénéité des données d'entrainement

Establishing a New State-of-the-Art for French Named Entity Recognition

We explore convert the NER annotations of the French TreeBank to a more user-friendly format and establish a new state of the art for French NER.

Pedro Ortiz Suarez, Yoann Dupont, Benjamin Muller, Laurent Romary, Benoît Sagot

Establishing a New State-of-the-Art for French Named Entity Recognition

French Contextualized Word-Embeddings with a sip of CaBeRnet: a New French Balanced Reference Corpus

We investigate the impact of different types and size of training corpora on language models.

Murielle Popa-Fabre, Pedro Ortiz Suarez, Benoît Sagot, Éric de la Clergerie

French Contextualized Word-Embeddings with a sip of CaBeRnet: a New French Balanced Reference Corpus

How OCR Performance can Impact on the Automatic Extraction of Dictionary Content Structures

We explore the impact of the OCR quality on grobid-dictionaries models.

Mohamed Khemakhem, Ioana Galleron, Geoffrey Williams, Laurent Romary, Pedro Ortiz Suarez