Frozen or Fine-tuned? Analyzing Deep Learning Models and Training Strategies for Optimizing Big Five Personality Traits Prediction from Text

Abstract

In today's digital age, the comprehension and prediction of human personality traits have assumed paramount significance. This study embarks on the task of forecasting the Big Five personality traits through textual data, harnessing the capabilities of advanced natural language processing models. The focal dataset is the ChaLearn First Impressions V2, a treasure trove of human-generated text coupled with Big Five personality trait labels. A diverse array of models undergo scrutiny, ranging from basic deep learning models like Deep Pyramid Convolutional Neural Network (DPCNN) and Hierarchical Attention Network (HAN) to cutting-edge transformer-based architectures such as BERT and FLAN-T5. These models undergo meticulous evaluation across various training scenarios, spanning scenarios where all layers are fine-tuned, only the embedding layer is freezed, and the complete layer freezing, with exclusive attention to Transformer models. Notably, models such as DPCNN and HAN emerge as stars, boasting remarkable accuracy attributable to their prowess in hierarchical feature extraction. Conversely, Transformer models like ELECTRA shine when layers remain frozen, showcasing their exceptional contextual comprehension. Furthermore, the study employs word clouds to visually encapsulate the essence of each Big Five personality trait, unraveling intricate relationships between specific words and these traits. The findings underscore the intricate interplay among model architecture, training methodologies, and layer freezing, offering valuable insights into strategies that yield optimal performance in predicting personality traits. In an age dominated by digital communication, this research contributes significantly to our understanding and prediction of human personalities. ©2024 IEEE.