Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models - Institut Jean-Nicod Accéder directement au contenu
Pré-Publication, Document De Travail (Preprint/Prepublication) Année : 2024

Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models

Résumé

Large Language Models have recently been applied to text annotation tasks from social sciences, equalling or surpassing the performance of human workers at a fraction of the cost. However, no inquiry has yet been made on the impact of prompt selection on labelling accuracy. In this study, we show that performance greatly varies between prompts, and we apply the method of automatic prompt optimization to systematically craft high quality prompts. We also provide the community with a simple, browser-based implementation of the method at https://prompt-ultra.github.io/ .
Fichier principal
Vignette du fichier
2407.10645v1.pdf (617.3 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04657856 , version 1 (22-07-2024)

Identifiants

Citer

Louis Abraham, Charles Arnal, Antoine Marie. Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models. 2024. ⟨hal-04657856⟩
12 Consultations
9 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More