Challenging the illusion of objectivity: an in-depth analysis of the preselected items evaluation (PIE) method in translation evaluation

Abstract

Purpose: The primary objective of this research paper was to examine the objectivity of the preselected items evaluation (PIE) method, a prevalent translation scoring method deployed by international institutions such as UAntwerpen, UGent and the University of Granada. Design/methodology/approach: This research critically analyzed the scientific and theoretical bottlenecks associated with the PIE method, specifically focusing on its parameters, namely the p-value and d-index, in adherence to established statistical protocols. Proposed remedies to mitigate the identified bottlenecks and augment the efficacy of the method were grounded in practicality. Findings: The paper provided an extensive overview of the PIE method, which served as the foundation for the subsequent analysis and discussions. This research presented potential avenues for refinement and contributed to the current debate on objective translation assessment by addressing the theoretical and practical challenges associated with the PIE method. Research limitations/implications: Translation researchers, practitioners and international institutions seeking to enhance the accuracy and reliability of translation evaluation should consider the implications of this research’s findings. Originality/value: Although several publications focused on the role of the PIE method in translation evaluation, no study(ies) is available to critically analyze the scientific and theoretical bottlenecks of this translation evaluation method. © 2024, Emerald Publishing Limited.