Avaluació de la traducció automàtica neuronal de documents judicials: estudi de cas de la traducció d'una ordre de detenció preventiva de l'espanyol a l'anglès
| Autor | Francisco J. Vigier-Moreno, Lorena Pérez-Macías |
| Cargo | Universidad Pablo de Olavide/Universidad Autónoma de Madrid |
| Páginas | 73-91 |
REVISTA DE LLENGUA I DRET #78
JOURNAL OF LANGUAGE AND LAW
ASSESSING NEURAL MACHINE TRANSLATION OF COURT DOCUMENTS: A CASE
STUDY ON THE TRANSLATION OF A SPANISH REMAND ORDER INTO ENGLISH
Francisco J. Vigier-Moreno∗
Lorena Pérez-Macías∗∗
Abstract
The court translation sector is showing considerable growth in demand due to the increasing number of proceedings
involving people who do not speak the language used by the authorities, and particularly across the European Union
(EU) since the passing of recent legislation that has enshrined the right to translation of essential documents in criminal
proceedings (Brannan, 2017). For the translation of legal texts, machine translation (MT) is not viewed with optimism due
to its disregard for the purpose and recipient of the translation (e.g., Wiesmann, 2019; Roiss, 2021), despite its potential
for saving time and the benets it offers at the terminological and phraseological levels (Killman, 2014) or at the level
of syntax (e.g., Heiss & Soffritti, 2018; Mileto, 2019; Wrede et al., 2020). The aim of this article is to discuss whether
translators can benet from MT when engaging in the challenging yet highly in-demand activity of court translation.
This article assesses the quality of English translations of a Spanish remand order produced by three different neural
machine translation (NMT) systems (DeepL, eTranslation, and Google Translate), using TAUS evaluation guidelines.
Keywords: neural machine translation; legal translation; court documents; remand order.
AVALUACIÓ DE LA TRADUCCIÓ AUTOMÀTICA NEURONAL DE DOCUMENTS
JUDICIALS: ESTUDI DE CAS DE LA TRADUCCIÓ D’UNA ORDRE DE DETENCIÓ
PREVENTIVA DEL CASTELLÀ A L’ANGLÈS
Resum
Al sector de la traducció judicial s’està observant un creixement considerable de la demanda a causa de l’augment del
nombre de procediments amb persones que no parlen la llengua utilitzada per les autoritats, especialment a la Unió
Europea (UE) des que s’ha aprovat la legislació recent que ha consagrat el dret a la traducció de documents essencials
en els procediments penals (Brannan, 2017). En el cas de la traducció de textos jurídics, la traducció automàtica (TA)
no es veu amb optimisme pel seu menyspreu al propòsit i el destinatari de la traducció (per exemple, Wiesmann, 2019;
Roiss, 2021), malgrat el seu potencial per estalviar temps i els avantatges que ofereix pel que fa a la terminologia i
la fraseologia (Killman, 2014) o la sintaxi (per exemple, Heiss i Soffritti, 2018; Mileto, 2019; Wrede et al., 2020).
L’objectiu d’aquest article és analitzar si els traductors poden aprotar la TA quan aborden l’activitat exigent però
molt demandada de fer traduccions judicials. En aquest article s’avalua la qualitat de les traduccions a l’anglès d’una
ordre de detenció preventiva en castellà produïdes per tres sistemes de traducció automàtica neuronal (TAN) diferents
(DeepL, eTranslation i Google Translate) seguint les directrius d’avaluació de la TAUS.
Paraules clau: traducció automàtica neuronal; traducció jurídica; documents judicials; ordre de detenció preventiva.
∗ Francisco J. Vigier-Moreno, Universidad Pablo de Olavide, fvigier@upo.es, 0000-0002-0996-2578
∗∗ Lorena Pérez-Macías, Universidad Autónoma de Madrid, lorena.perez@uam.es, 0000-0003-4361-1250
Article received: 15.07.2021. Blind reviews: 13.09.2021 and 28.01.2022. Final version accepted: 19.10.2022.
Recommended citation: Vigier-Moreno, Francisco J., & Pérez-Macías, Lorena. (2022). Assessing neural machine translation of
court documents: A case study on the translation of a Spanish remand order into English. Revista de Llengua i Dret, Journal of
Language and Law, 78, 73-91, https://doi.org/10.2436/rld.i78.2022.3691
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 74
Contents
1 Translation of court documents with an emphasis on Spain
2 MT and legal texts
3 Case study: Machine-generated translations into English of a Spanish remand order
3.1 Characteristics of Spanish remand orders and their translation
3.2 Methodology
3.2.1 Selected NMT engines
3.2.2 MT output error categorisation
3.3 Results
4 Conclusions
5 References
6 Appendix (Spanish remand order)
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 75
1 Translation of court documents with an emphasis on Spain
The number of court proceedings involving individuals who do not speak or do not have sufcient prociency
in the language used by the court authorities has been steadily increasing in recent decades worldwide. In
turn, this increase has led to a concurrent rise in the demand for court translation and interpreting, especially
in criminal proceedings, in which translating and interpreting services play a crucial role in guaranteeing
access to fundamental rights such as the right to a fair trial (Ortega Herráez et al., 2013, p. 89). Accordingly,
there has been a steady increase in the need to raise awareness “of the importance and utility of high-quality
court-related translation services” (Killman, 2021, p. 76).
Nevertheless, the practice of court translation was not being provided consistently or systematically across EU
Member States for documents that allowed defendants to fully participate in the proceedings (Brannan, 2017,
p. 44). In response, the EU has enacted legislation to reinforce language assistance in courts, especially in the
eld of criminal proceedings. Hence, EU legislation (e.g., Directive 2010/64/EU of the European Parliament
and of the Council of 20 October 2010, on the right to interpretation and translation in criminal proceedings)
has enshrined the right to translation in criminal proceedings, with specic reference being made to essential
documents such as indictments, judgements, and any court decisions that entail depriving a person of their
liberty. Emphasis has also been placed on the quality of these translations.
Like all other EU Member States, Spain has transposed this supranational legislation into its national law
(Vigier-Moreno, 2020, p. 36). Pursuant to this new Spanish legislation, defendants who neither speak nor
understand Spanish or another of the ofcial languages in which proceedings may be conducted are entitled
to receive a written translation of the documents that are deemed essential to ensure their right to self-defence
in court, with express reference to court decisions involving imprisonment of the defendant, indictments, and
judgements. Since passing this legislation, court translation has gained prevalence in criminal proceedings
held in Spain and must now be taken into account as an important element of the legal process. In 2021,
the cost of translation and interpreting services for legal proceedings totalled more than €9 million (Spanish
General Council of the Judiciary, 2021). Previously, the defendant was frequently not provided with written
translations of these documents, but rather was informed of their content through sight translation alone, often
in a summarised version (Ortega Herráez, 2013, p. 13). However, since this legislative update, the Spanish
Supreme Court and the Spanish Criminal National Court (Audiencia Nacional) have already issued rulings
on appeals based on the failure to provide a translation of documents such as police reports, lending further
evidence of future increases in court translation workload (Izquierdo Valverde, 2016, p. 23).
In both the court translation sector and the legal translation market more generally, translation requests
regularly involve tight deadlines and budget constraints (Pasteur, 2013; Killman, 2021). This is particularly
the case when translations are needed in the investigative stage1 of criminal proceedings; clearly dened
timeframes for certain judicial procedures can substantially compress the requested turnaround times (Aldea
Sánchez et al., 2004). Moreover, many translators providing translations for Spanish courts have to translate
in both directions. As a result, court translators who are native speakers of the language used by the court
authorities may often have to translate into the other language (Feria García, 1999; Ortega Herráez, 2013;
Nauen, 2020), which is generally perceived as more challenging than translating into one’s native tongue
(Dubĕda, 2021).
The majority of court documents requiring translation in Spanish courts are produced by legal professionals
(e.g., judges, prosecutors, defence lawyers) and include a good deal of legal terminology and phraseology
(Ortega Arjonilla, 2012). These texts also exhibit typical features of legal language, which may involve
complex syntax or highly convoluted sentence structures, and may even have faulty or decient formulations
(Ortega Herráez, 2013, p. 19). In light of these challenges, the translation of court texts has been characterised
as a challenging undertaking, particularly in terms of legal asymmetry (Prieto Ramos, 2014), since court
documents are anchored to the legal system from which they originate and reect procedural specicities that
may not coincide, at the conceptual and terminological level, with the legal system(s) of the target language
(Casamayor Maspons, 2020). However, like many other legal genres, court documents tend to follow a
1 In the specic case of Spain, pre-trial investigative proceedings are carried out by an investigating judge in the juzgados de
instrucción or investigative criminal courts (Granados Meroño & Orts Llopis, 2021, p. 95).
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 76
standard structure, with frozen language and repetitive patterns. These consistent textual features may facilitate
the task as court translators become more experienced (Nauen, 2020), and potentially support the use of
translation technologies. In the case of translation memories, translators can make use of previously completed
translations as they translate similar texts. As far as machine translation (MT) is concerned, translators can
make use of translations that MT provides by drawing on similar or related sources of text in its database
(see Section 2).
MT is already being used for different purposes in legal settings worldwide, from EU institutions (Biel, 2017)
to national police settings (Keaton, 2020). We therefore consider it relevant to assess the quality and usefulness
of MT output in the underexplored eld of court-related translation assignments. This study investigates
whether court translators may benet in different ways from the use of MT by analysing the quality of the
English translations of a Spanish remand order produced by three neural machine translation (NMT) systems
(DeepL, eTranslation, and Google Translate) in accordance with TAUS evaluation guidelines. The analysis
assesses the raw MT output with an eye to the purpose of the translation of an essential court document,
which in the current study is to safeguard due process of the English-speaking person involved in the court
proceedings.
2 MT and legal texts
From its beginnings, MT has been studied from different disciplines such as computational linguistics or
computer science. Early views of MT sought to completely replace humans with machines (Abaitua Odriozola,
1999), while more recent perspectives seek to reconcile this dichotomy to achieve a balance between the
capabilities of the machine and those of the human translator (an overview of this evolution can be found in
Alonso Jiménez & Calvo Encinas, 2015).
NMT systems are based on a network structure capable of analysing a sentence and suggesting a translation,
using probability criteria based on this analysis (Neubig, 2017). However, in order to increase the probability of
achieving an ideal translation, this type of system needs to be trained with new input, and the reference corpus
needs to be progressively updated for both source and target languages (Cho et al., 2014). NMT systems are
not developed with a set of dened functionalities and instructions, nor are they designed to solve a specic
problem mechanically or digitally. In contrast, these systems are developed to enable them to be taught to
solve problems on the basis of examples. The results are gradually improved with time and training, and
solutions may vary according to different contexts.
In the legal translation context, studies have assessed the quality of several MT systems in different language
pairs and ways. Killman (2014) evaluated MT output from Spanish to English in the Spanish judicial context,
focusing on a sample of more than 600 terms and phrases from a civil judicial text of judgment summaries
from the Spanish Supreme Court. The output provided by Google Translate (GT), at the time a statistical
MT system, was assessed as accurate in almost 65% of the sample, which was seen as a positive result in an
area where the translation of terminology is often perceived as difcult. Statistical MT errors were especially
prevalent at the terminological and phraseological levels in cases that involved lexical ambiguity or needed
to be rendered in the target language in a contextually specic way.
Mileto (2019) also observed advantages at the terminological level in her study, in which three different MT
engines (MT@E, GT, and SDL Language Cloud) were examined. Specic ndings included appropriate
translations of collocations and terminological consistency. Wrede et al. (2020) analysed the effectiveness of
L2 post-editing (PE) from Slovak into German in the legal eld with a case study in which a group of students
were asked to compare the quality of a post-edited GT-generated translation with the human translation of the
same source text. In a study conducted by Vigier-Moreno and Pérez-Macías (2020), participants were required
to complete an assessment of MT quality with and without pre-editing in the case of three different free NMT
engines (GT, DeepL, and the EU Council Presidency Translator) translating from Spanish into English, and
there was consensus on the usefulness of MT in solving vocabulary and terminology challenges when working
into a non-native language. Roiss and Zimmermann González (2020) found that DeepL was a valuable tool for
L2 translation, as it offered several translation options for the target text, as well as lexicographical information
on the terms in the source language.
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 77
In contrast to studies investigating the utility of MT use in legal contexts, Heiss and Soffriti (2018) conducted
a study to examine the impact of the use of DeepL on university training in specialised translation from
Italian into German. The study addressed three different subject areas (legal texts, university institutional
information, and technical instructions), and the results provided by DeepL for each of the texts in these areas
were evaluated using BLEU, “an inexpensive automatic evaluation that is quick, language-independent, and
correlates highly with human evaluation” (Papineni et al., 2002). Similar scores were obtained for the legal
and technical domains, while a signicantly lower score was recorded for university institutional information.
Wiesmann (2019) set out to assess the level of accuracy of MT for translating legal texts from Italian into
German for the purpose of determining to what extent MT should have a place in legal translation pedagogy.
The MT engines she assessed were DeepL and MateCat, a CAT tool that, at the time of the study, included
DeepL, the NMT version of GT, and Microsoft Translator, which was still a statistical system. Despite nding
appropriate or nearly appropriate MT suggestions in her uency and accuracy assessment of DeepL and
MateCat, Wiesmann (2019) concluded that the overall quality was not sufcient to give PE more prominence
in the legal translation classroom. Vigier-Moreno and Pérez-Macías (2020) found similar challenges with
respect to PE effort in the use of MT.
In its review of the above-cited literature on MT in the legal eld, the present study seeks to better understand
MT output quality by differentiating four categories (terminology, accuracy, uency, and style) when MT is
used to translate a legal text from Spanish to English. The study also includes an assessment of the severity of
the errors detected, which may also improve understanding of the strengths and weaknesses of current NMT
engines when applied to legal translation. The types of errors and their severity are particularly salient to this
discussion, since results related to these categories can be interpreted in relation to questions of MT quality.
3 Case study: Machine-generated translations into English of a Spanish remand order
This case study evaluates the quality of NMT applied to a specic language combination (Spanish to English)
and a specic type of text in the legal eld, i.e., a remand order, an essential document in Spanish criminal
proceedings. Firstly, following the integrative model put forward by Prieto Ramos (2014), the translation
assignment is described to determine the purpose of the translation itself and the specic legal communicative
coordinates that allow for adequacy in legal translation problem-solving. Secondly, the source text is analysed
from a genre perspective to show the way in which certain textual aspects condition the translation strategy
adopted by the translator. There then follows a description of the methodology used in this study and discussion
of the results obtained after assessing the quality of different sources of NMT-generated output in accordance
with TAUS evaluation guidelines, with an emphasis on elements identied by previous research.
3.1 Characteristics of Spanish remand orders and their translation
Remand orders are essential court documents that serve the specic purpose of implementing a court’s
decision to hold a defendant in custody until a trial takes place. Translations of these documents remain an
important means by which non-Spanish speaking defendants are enabled to read and understand the text
throughout the criminal proceedings. Moreover, translations of these texts enable a defendant to exercise their
procedural rights (e.g., self-defence), insofar as they provide sufcient access to these performative texts that
gure into the overarching due process and procedural safeguards of the legal system. While translations of
these texts are largely informative in nature in that they do not produce the same legal effect as the Spanish
language original, nonetheless the translations provide necessary information in the defendant’s language that
signicantly enhances their understanding of the proceedings.
For the purposes of this case study, the source text is a Spanish auto de prisión provisional. For the sake of
simplicity, we will refer to this text in English as a remand (in custody) order, a legal instrument under English
law that can be considered a functional equivalent (Oxford University Press, n.d.). The specic remand order
under investigation was retrieved from the public website of the Centre for Judiciary Documentation of the
Spanish General Council of the Judiciary,2 and is a 2-page, 876-word document (see Appendix3). This text
2 A copy of the remand can be found on the Centre for Judiciary Documentation website.
3 Details (personal, geographical, or otherwise) have been redacted to preserve anonymity.
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 78
is relatively short in comparison to other court documents. To our knowledge, remand orders have received
scarce attention from translation studies scholars, despite their procedural importance in legal settings.
Previous studies by translation scholars on the umbrella genre auto (interlocutory court order) are limited
in comparison with, for example, the abundance of translation-specic studies focused on judgements (e.g.,
McAuliffe, 2013; Orts Llopis, 2017). To our knowledge, only two studies have addressed the translation of
Spanish autos. Garofalo (2009) studied the translation into Italian of an auto de procesamiento por delito
de robo y agresión sexual (i.e., an indictment for robbery and sexual assault) and an auto acordando el
sobreseimiento libre por exención de la responsabilidad del procesado (i.e., an order of dismissal due to
exemption of the defendant from criminal liability), while Granados Meroño and Orts Llopis (2021) addressed
the translation into English of an auto de sobreseimiento provisional (i.e., a provisional order of dismissal)
in a political corruption case.
Scholars such as Orts Llopis (2018) and Monzó-Nebot (2020) have articulated the importance of studying
legal genres for the purposes of legal translation. By extension, a characterisation of the genre auto de prisión
provisional provides legal translators with insight into the legal mechanisms and textual features of these
documents in this specic communicative situation, assisting them to better adapt their translation to the
intended environment. According to Jowers (2016), the term auto refers to a specic type of court decision
which generally resolves interlocutory issues within court proceedings. As explained by Garofalo (2009) and
Granados Meroño and Orts Llopis (2021), all autos share a common macrostructure, which consists of the
following sections: rst, the encabezamiento (heading), which presents the parties involved and the object of
the proceedings; second, the antecedentes de hecho (factual background), which provide a description of the
facts and the legal procedures that have taken place thus far; third, the fundamentos de derecho (legal grounds),
which provide an account of the law that must be applied to make a decision in view of the facts; fourth, the
parte dispositiva (operative part), which includes the decision on the matter; and fth, the diligencia nal
(nal certication), whereby the court registrar certies the document.
A macro-contextualisation of our auto de prisión provisional can be further elaborated. Dened as “a court
decision ordering the precautionary measure of provisional imprisonment” (Real Academia Española, 2022)
and regulated under section 502 of the Spanish Code of Criminal Procedure, this document must state the
reasons for ordering a measure as exceptional as imprisonment, taking all the factors involved into account
(on the one hand, the freedom of a person presumed innocent; and on the other, the administration of criminal
justice and prevention of criminal activity). Therefore, remand orders are only produced in the pre-trial stage of
criminal proceedings, during which the investigating judges in a case adopt medidas cautelares (precautionary
measures), such as ordering the defendant to be remanded into custody, to be released with or without bail,
or to stay away from the victim or a witness.
Our remand order has the same macrostructure of an auto as described above. However, a more thorough
analysis leads to the identication of the following elements. First, the heading gives information about the
court (juzgado de instrucción or investigating court) and its location, the type of proceedings (identied as
diligencias previas4) and the name of the document (auto). Second, the factual background section (in our
text referred to as hechos) comprises two numbered paragraphs. The rst of these relates the facts leading to
the initiation of the proceedings (according to which the defendant is alleged to have killed a man, taken his
belongings, and demanded a ransom from the victim’s family). The second paragraph describes the motion
made by the prosecution (that the defendant be remanded into pre-trial custody). Third, the section on the
legal grounds (here designated as razonamientos jurídicos) consists of three numbered subsections. The
rst and longest of these subsections refers to the applicable law and requirements for pre-trial custody to
be ordered. The second subsection species that the requirements are met in this specic case, i.e., that the
defendant is believed to be criminally liable for certain offences and that legal procedure has been followed
4 The term diligencias previas refers to the proceedings held during the investigative stage of the procedimientos abreviados
(abbreviated proceedings), a specic type of criminal proceedings established by Spanish law to prosecute offences which may lead
to imprisonment sentences of no more than nine years, and is by far the most common in Spanish criminal justice. These pre-trial
proceedings are held in the investigating court, whereas the trial takes place in either the juzgado de lo penal or criminal trial court
(if the penalty sought entails a custodial sentence of less than ve years) or in the audiencia provincial or provincial court (if the
sentence requested by the prosecution involves between ve- and nine-years’ imprisonment).
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 79
(namely, a hearing of a motion made by the prosecutor). Finally, the third subsection sets forth that it is
appropriate for the judge to order that the defendant be remanded into custody and species the conditions of
such imprisonment. Finally, the parte dispositiva describes the judge’s main decision regarding custody and
other measures, followed by the court registrar’s certication.
A micro-textual analysis of our remand order reveals typical features of Spanish legal discourse (Alcaraz Varó
et al., 2009, pp. 24–32), such as the use of grandiloquent and very formal language, seldom-used verb tenses
(e.g., passive imperatives like líbrese), prepositional phrases (e.g., conforme a lo dispuesto en), convoluted
syntax (one-sentence paragraphs), abuse of the gerund (e.g., quitándole acto seguido sus pertenencias), and
frozen language (e.g., así lo acuerda, manda y rma). At the lexical and terminological level, there is “a
spectrum ranging from near-zero difculty, for the terms that have the ‘exact’ legal/linguistic correspondence
in the source and target legal systems, to near ‘untranslatability’, for those terms that are so specic” (Scarpa
et al., 2014, p. 68). In other words, not only does the text feature a number of legal terms that can be more or
less easily replaced in the target text with functionally-equivalent counterparts (e.g., audiencia, delito, prisión
and pena), but also Spanish legal system-bound terms (e.g., names of courts like Audiencia Provincial, legal
professions like Letrado de la Administración de Justicia) the translation of which relies on an approach based
on the communicative situation (Prieto Ramos, 2014).
A pre-translation analysis enables the translator to make translation decisions that will help them achieve
their translation objective more successfully. For instance, a recipient-oriented approach to translation aids
the translator in choosing how to render specic textual elements in translation. As Chromá (2016, p. 78)
remarks, “if it is clear that the translated text is to be used by a recipient in a particular (English-speaking)
country, the translator may choose a relevant variety of legal English at least by selecting proper legal
terminology used in the translation” (Chromá, 2016, p. 78). In the specic case treated in the present case study
(see Appendix), information related to the defendant shows that English is spoken in his country of origin,
chiey as a lingua franca. As such, translation decisions will likely be guided by the possibility that English
is used by the defendant as a lingua franca and, consequently, a variety of English that is widely understood
would be appropriate. This decision will also likely take into account the fact that the defendant is not a legal
professional; as far as possible, therefore, the translator will need to render the Spanish text into an English
version capable of being understood by English speakers from a range of backgrounds.
The above recommendation follows the same approach used to translate Spanish indictments in an earlier
study (Vigier-Moreno, 2020), namely, to produce a target text in a variety of English that is as universal
as possible and not tied to any specic English-speaking legal system. The writing style, while technically
precise, remains concise, grammatically correct, and attentive to the needs of the reader. Furthermore, since
the main purpose is that of properly informing the defendant “of the content and sense of the source legal text”
(Chromá, 2016, p. 79) rather than creating a document with the same legal force as the original, translation
techniques that aid the recipient to understand the Spanish legal system are likely to be of use.
Automated approaches to translation fail to take into consideration these aspects (i.e., a recipient-oriented,
functionalist approach), despite their importance in legal translation decision-making, as has been discussed
(Wiesmann, 2019; Roiss, 2021). With this in mind, we assessed the quality of output generated by NMT
engines of the Spanish remand order to determine whether translations generated by current neural MT systems
are of limited utility for court translators, or whether, on the contrary, court translators can benet from the
use of MT in their professional practice.
3.2 Methodology
We conducted an exploratory-descriptive study to determine whether translators can benet from raw NMT
output when translating a complicated court document from Spanish to English. This study focuses on the
divergent approaches adopted by human and machine translation. On the one hand, professional translators
are likely to make macro-level decisions based on situational aspects of the translation assignment while, on
the other, MT systems generate translations based on sources of co-text. In this article, therefore, we assess
English translations of a Spanish remand order obtained from three MT systems (DeepL, eTranslation, and
GT) to test the effectiveness of MT in this eld. The quality of the output from these systems was analysed
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 80
using TAUS evaluation guidelines – criteria developed to examine MT systems – while also taking into account
the purpose of the translation. At this point, ethical questions arise as to whether this type of text should be
translated using MT engines, since the content of the documents will be shared with the MT providers, which
will subsequently use that data input to train and improve their engines. However, not only is our remand order
available online for anyone to read, it was also edited before being machine-translated (all personal details
contained in the remand order were replaced with ctitious ones and then redacted from the Appendix) to
preserve the condentiality of the case.
3.2.1 Selected NMT engines
The present study involves GT and DeepL NMT engines. In 2006, GT was initially launched as a statistical
system. At the end of 2016, GT was transformed into an NMT engine that currently offers more than 100
different languages and is used by over 500 million people a day (Sommerlad, 2021). DeepL, also a general
purpose, free online system, was launched in 2017 and currently offers translations for 24 different languages,
including the option to select American or British English. The third MT system in this study is eTranslation,
an NMT system provided by the European Commission since 2017, when the EC retired their statistical
system MT@EC. eTranslation was created for staff and translators working for EU institutions or agencies
and can also be used by small and medium-sized enterprises and universities in the EU, Iceland, and Norway
(European Commission, 2021). This NMT engine draws on Euramis (European advanced multilingual
information system), a multilingual corpus including aligned legislative documents in the 24 languages
(European Commission, 2021). For this reason, eTranslation could initially be expected to produce more
favourable results, as it is mainly based on multilingual EU administrative and legal documents and could
therefore be considered more legal-specic than the two other open-domain systems in this study, i.e., GT
and DeepL.
3.2.2 MT output error categorisation
Once the different outputs were generated for each of the selected NMT engines in April 2021, the quality of
output from these systems was evaluated according to TAUS guidelines (TAUS, 2021), which include the error
types shown in Table 1. For considerations of space, only the types of errors found in the output generated
by the NMT engines used in this study are listed.
Each of the errors detected in the texts generated by the MT engines was assigned a severity level. For the
sake of simplicity, the levels proposed by TAUS were simplied into major and minor. Minor errors are those
that affect stylistic quality, uency or clarity, but do not detract from the meaning; while major errors may
confuse or mislead the user, or hinder the appropriate use of the translation by involving a signicant change
in meaning or occurring in a visible or important part of the content (TAUS, 2021).
3.3 Results
Table 1 summarises the analysis conducted on the raw MT output generated by each of the engines. The most
prominent errors identied in the translations generated by the MT engines relate to (1) terminology, an error
category assigned to instances in which “a term (a domain-specic word) is translated with a term other than
the one expected for the domain”; and (2) accuracy, which occurs when “the target text does not accurately
reect the source text” (TAUS, 2021). Other errors were identied with respect to accuracy, uency, and
style, but for considerations of space, these terminological issues will be the primary focus of our discussion.
Table 1. Errors according to typology and severity
Error type
DeepLAmEn
(927 words)
DeepLBrEn
(916 words)
eTranslation
(880 words)
GT
(899 words)
Accuracy (total) 8 10 29 21
Omissions 1 3 3 1
Mistranslations 5 423 15
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 81
Error type
DeepLAmEn
(927 words)
DeepLBrEn
(916 words)
eTranslation
(880 words)
GT
(899 words)
Over-translations 1 1 1 1
Under-translations 1 1 1 4
Untranslated -11-
Fluency (total) 8 8 7 9
Punctuation 3 3 2 4
Spelling 2 2 -1
Grammar 1 1 42
Inconsistency 2 2 1 2
Terminology (total) 21 25 30 35
Inconsistent use of
terminology 2 3 2 5
Style (total) - 1 5 7
Inconsistent style - - 3 -
Unidiomatic -127
TOTAL ERRORS 37 44 71 72
SEVERITY COUNT 15 minor/
22 major
16 minor/
28 major
27 minor/
44 major
34 minor/
38 major
The translation of legal system-bound terms presents a challenge for generalist NMT systems, since
extralinguistic context is beyond the purview of natural language processing with computers. (N)MT systems
therefore cannot be expected to take into account aspects such as the purpose or the recipient of the translation
(Killman, 2015). Our study corroborates this understanding insofar as the output generated by the NMT
engines sometimes provided unsuccessful translation options into English for Spanish legal system-bound
terms. Table 2 illustrates the translations produced for some of the names of courts, court staff, and legislation
pieces found in the source text; where the NMT system offered multiple translations in the target language
version for a given unit of the source term at different places in the text, the various translations are shown
separated by a slash. The primary issue with the translations provided by the MT engines was that they
tended to offer over-literal options including calques and false cognates, which could in turn lead to changes
in meaning or even nonsensical output.
Table 2. English translations of Spanish legal system-bound terms
Source Text DeepLAmEn DeepLBrEn eTranslation GT
Código Penal Criminal Code /
Penal Code
Criminal Code /
Penal Code Criminal Code Penal Code
Juzgado de
instrucción Court of Instruction
Court of Instruction
/ Examining
Magistrate’s Court
Court of Inquiry
/ Court of
Investigation
Instruction Court /
Court of Instruction
Letrado de la
Administración de
Justicia
Counsel for the
Administration of
Justice
Counsel for the
Administration of
Justice
Counsel for the
Administration of
Justice
Letter for the
Administration of
Justice
Magistrado-Juez Magistrate-Judge Magistrate-Judge Magistrate-Judge Magistrate-Judge
Ministerio Fiscal Public Prosecutor ’s
Ofce
Public Prosecutor’s
Ofce
Public Prosecutor’s
Ofce / Ministry
Public Prosecutor’s
Ofce / Public
Prosecutor
For example, the terms juzgado de instrucción, magistrado, letrado de la Administración de Justicia and
ministerio scal give rise to some interesting inadequacies in the output. As described previously, the juzgado
de instrucción is characteristic of the Spanish criminal court system in that it undertakes pre-trial investigations;
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 82
hence, the calque translations court of instruction and instruction court and court of inquiry5 are likely to
confuse the reader as to the role and functions of the Spanish court. Examining Magistrate’s Court may work
from a functional perspective, but the differences between a magistrates’ court and a juzgado de instrucción
are so great (e.g., in England, a magistrates’ court is normally made up of three lay people who are neither
legally qualied nor remunerated) that it is not advisable to use this term; court of investigation seems to be
a more adequate solution. Another issue is that two engines (DeepLBrEn and eTranslation) offered different
English translations of this Spanish court within the same text, which could potentially cause confusion.
As far as the names of judicial actors are concerned, the Spanish term magistrado-juez refers to a member of
the judiciary with a higher rank than that of juez (Real Academia Española, 2022) who sits on a single-judge
court rather than on a panel, as magistrados would usually do; therefore, the calque translation offered by
all four MT engines (magistrate-judge) can convey a different meaning, since magistrate in many English-
speaking systems refers to a judicial ofcer with limited jurisdiction in minor criminal cases and sometimes
without a legal qualication. As for the term Letrado de la Administración de Justicia, both DeepL and
eTranslation offered Counsel for the Administration of Justice, a calque translation that still conveys accurate
meaning, though a more uent and functional rendition, given its marginal signicance in the source text,
would be registrar or court clerk. The term Ministerio Fiscal was mainly translated as Public Prosecutor’s
Ofce, even though the term in Spanish is used metonymically in some parts of the text and may more
appropriately have been translated as Public Prosecutor or Public Prosecution. Regarding the translation of the
names of legislative texts, the English translations do not present problems in terms of adequacy or accuracy,
but they are used inconsistently within the text, and this may degrade the overall quality of the translation.
Other salient inadequate translations generated by the MT engines relate to legal terms that initially might
appear to have a more straightforward functional translation in the target language. However, some of these
terms are highly signicant to the case in hand and have nuances that are deeply rooted in the Spanish legal
system. Consequently, the rendering of these terms in another language requires the translator to conduct in-
depth legal comparative analyses to ascertain whether the differences in concepts between the two languages
and legal systems are not decisive and functional equivalents can be used. If differences are identied, “the
use of presumed equivalents is risky, as it can trigger erroneous associations and interpretations” (Fuglinszky
& Somssich, 2020, p. 754). In light of the translations offered for some of these key legal terms (see Table 3), the
systems once again seem to adopt an over-literal approach to translation, which may entail serious changes
in meaning.
Table 3. Translation of key Spanish legal terms without a straightforward functional equivalent
Source Text DeepLAmEn DeepLBrEn eTranslation GT
auto auto order car car
auto de prisión prison order prison sentence prison order arrest warrant
detención ilegal illegal detention illegal detention illegal detention illegal detention
investigado
the accused
/ defendant /
investigated person
/ person under
investigation / the
investigated
the accused /
person under
investigation
the accused / the
accused person /
investigator / the
person investigated
/ the person under
investigation / the
investigated person
the accused /
person under
investigation / the
investigated person
/ the investigated
prisión provisional provisional
imprisonment
provisional
detention
pretrial detention /
arrest
provisional
imprisonment
/ provisional
detention
5 Court of inquiry is dened as “a group of people who are ofcially appointed to investigate a serious accident or incident, or an
ofcial investigation into a serious accident or incident”.
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 83
Source Text DeepLAmEn DeepLBrEn eTranslation GT
prisión provisional
comunicada y sin
anza
provisional
imprisonment
without bail
/ provisional
commanded prison
without bond
provisional
detention without
bail / provisional
commanded prison
without bond
provisional
detention
communicated
and without bail
/ provisional
communical prition
and without ance
provisional
imprisonment
communicated
and without bail /
provisional prison
communicated and
without bond
For instance, investigado, which literally translates as the investigated man, in our text refers to the person
being charged. Translations in our results include both the accused and the person under investigation, among
others. However, the translator must take into account the fact that, under Spanish procedural law, a person
charged in a criminal case is referred to as either investigado or acusado (among other designations) according
to the stage and type of court proceedings. The term investigado is used in our remand order because the
judge has commenced court proceedings against him and there is enough preliminary evidence to warrant
continuing the investigation. However, this designation will change if a formal indictment is issued against
the defendant later in the proceedings, such that he eventually stands open trial; at this stage the defendant
would be referred to as el acusado. From a procedural standpoint, it is therefore important that the translator
be aware of how investigado or acusado is being used to be able to translate these terms in a contextually
appropriate way. Other translations in our sample include defendant (an adequate functional equivalent) and
investigator (an obvious mistranslation).
As explained in Section 3.1, an auto is an interlocutory court order and, more specically, an auto de prisión
(provisional) is the document ruling that the defendant be remanded into (pre-trial) custody. The translations
offered by the MT engines vary considerably in the rst case, from order, an acceptable rendition, to the
contextually inappropriate car. In the case of auto de prisión, prison sentence (DeepLBr) and arrest warrant
(GT) – both completely different concepts – are provided. Something similar occurs with the Spanish term
prisión provisional, a crucial concept in the context of our case study. This term is conveyed into English
through options such as provisional imprisonment, provisional detention, and arrest, some of them offered as
alternatives within the English translation by the same NMT system. In English law, detention, for example,
refers to the deprivation of liberty of a person subsequent to arrest, and may normally last up to 36 hours
(Oxford University Press, n.d.). This English term would more accurately convey the Spanish term detención,
which also refers to the deprivation of liberty of an arrested person, and hence should not be used as an
expression of the term prisión provisional (see Section 3.1). Furthermore, in our document, the specic type
of pre-trial custody that the judge imposes on the defendant is prisión provisional comunicada sin anza, that
is, the defendant will be remanded into custody (prisión) pending trial (provisional) without bail (sin anza).
Notwithstanding, the defendant will be entitled to visitation and communication (comunicada) by phone
or through written correspondence with their defence attorney, a minister of their religion, a physician, and
relatives, among others. As shown in Table 3, the resulting translation options include mistranslations (e.g.,
commanded), calques (e.g., communicated), and nonsense (e.g., communical).
Finally, some of the terminological errors are nuanced. For example, the literal translation of detención ilegal
as illegal detention, which was offered by all four NMT systems, has a different context of use. According
to the Spanish Criminal Code, detención ilegal is dened as illegally locking up or detaining someone, thus
depriving them of their liberty; in our remand order, the term applies to the allegation that the defendant
restricted the victim’s freedom of movement. However, in the English-speaking legal domain, illegal detention
is normally used to describe a situation in which a law enforcement ofcer restricts a person’s freedom to
leave without legal justication (USLegal, n.d.). Bearing in mind that the defendant is not a member of law
enforcement, a more appropriate rendition would be a generic term such as deprivation of liberty, or a more
common legal term such as kidnapping.6
6 Under English law, kidnapping is dened as “carrying a person away, without his consent, by means of force, threats, or fraud”
(Oxford University Press, n.d.).
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 84
We also identied accuracy errors, which, according to TAUS and as previously described, occur when
translation options distort source text meaning. However, the distribution of these is very uneven, depending
on the MT engine. The most noticeable errors in this category include mistranslations caused by the use of
false friends. For example, resolución was translated as resolution and not as decision or ruling; asistencia
was translated into English as assistance, when attendance or presence would have been the more appropriate
translation; the verb acordar, which in Spanish legal discourse means to dictate or rule, was translated as
agree, according to its most frequent meaning in Spanish; partido, which refers to partido judicial (judicial
district), was translated by GT as party in the political sense. Other errors of accuracy include severe errors
in meaning and non-sensical renderings in the target text. For example, indiciariamente, which means
according to preliminary evidence, was variously translated as indiciously (in both DeepL versions), evidently
(eTranslation), and incidentally (GT). The expression doy fe, which is used in Spanish legal language in
attestation or certication clauses, was translated as doy faith and I give faith (eTranslation). Another example
of an inaccuracy error at the syntactic/sentential level is the translation provided by eTranslation (by means
of which the Ministry has been held, and that the Ministry has been held for the purpose of the arrest of
the Ministry) for the Spanish en concreto la ocupación de los efectos sustraídos, el registro del lugar que
habitaba, las llamadas exigiendo un rescate y lo declarado por el propio investigado, which introduces
grammatical incongruence and distorts the various relationships between various clauses established in the
source language text.
Despite these terminological and accuracy errors, the MT-generated translations, surprisingly, produced a much
lower number of uency and style errors (see Table 1). Whereas terminological and accuracy errors totalled
29, 35, 59, and 56 in the analysis of the translations generated by DeepLAmEn, DeepLBrEn, eTranslation
and GT, both uency and style errors in aggregate added up to only 8, 9, 12, and 16 respectively. This is
exemplied in the Spanish sentence shown below, followed by translations from DeepL, eTranslation, and GT:
Conforme a lo dispuesto en los artículos 503 y 505 de la Ley de Enjuiciamiento Criminal para que
proceda la prisión provisional se requiere: (source text)
In accordance with the provisions of articles 503 and 505 of the Criminal Procedure Law, in order
to proceed with the provisional imprisonment it is required: (DeepLAmEn)
In accordance with the provisions of articles 503 and 505 of the Law of Criminal Procedure, in order
for provisional detention to proceed, the following is required: (DeepLBrEn)
Pursuant to articles 503 and 505 of the Criminal Procedure Act, in order for pretrial detention to
proceed, the following are required: (eTranslation)
In accordance with the provisions of articles 503 and 505 of the Criminal Procedure Law, for
provisional detention to proceed, the following is required: (GT)
Despite faulty punctuation (i.e., missing punctuation after Criminal) and the unnatural word order (i.e.,
anteposition of the prepositional phrase and placing of the subordinate clause before the main clause) in the
source text, the sentence was adequately rendered into English by the NMT systems in our study. Furthermore,
this example also shows that NMT systems can successfully address the translation of frozen patterns that are
common in legal discourse, such as the Spanish preposition-based bundle conforme a lo dispuesto en, which
was conveyed as in accordance with (DeepL and GT) and pursuant to (eTranslation). The repetitive nature
of legalese therefore seems to conrm the potential benets of MT in legal translation.
In terms of uency, errors primarily included cases of poor punctuation (e.g., lack of commas), incorrect
or inconsistent spelling (Cordova/Córdoba – GT), poor collocations (e.g., parties involved of this order –
DeepLBrEn), and syntactic and prepositional calquing (hold criminally responsible for the said crime to [name
redacted] – eTranslation). As for style, the errors we identied mostly demonstrated a lack of idiomaticity in
the English translation, e.g., where he hit him on the head with a wood (DeepLBrEn) and where he hit him
with a wood on the head (DeepLAmEn).
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 85
It must also be acknowledged that certain translation options for very obscure and convoluted Spanish legal
expressions were surprisingly accurate. Such is the case for the fossilised language dedúzcase testimonio de
esta resolución. This expression, which refers to the inclusion of a certied copy of the court order in the
le, can be difcult to grasp for translators without a sound legal background or relevant experience in the
eld, especially in the imperative passive form used in our text. However, it was translated as a copy of this
order shall be drawn up by DeepLBrEn, a copy of this resolution shall be drawn up by DeepLAmEn and
bear witness to this resolution by eTranslation, all of which convey the sentiment of the source language
formulation. All these examples show how systems draw on corpora and attempt a phrase-based or even
sentential approach with different degrees of statistical accuracy.
Finally, in light of our assessment of the output produced by each NMT engine used in this study, an interesting
phenomenon has been observed with respect to the comparison between the two generalist systems used
(DeepL and GT) and the more domain-specic eTranslation (mostly fed with multilingual EU legislative
corpora). Contrary to expectation, eTranslation was not found to convey legal terminology more accurately
than GT or DeepL, and in fact scored lower in terms of terminology and accuracy.
4 Conclusions
Our TAUS-informed quality analysis of four NMT-produced English translations of a Spanish remand order
was conducted according to four broad categories: accuracy, uency, terminology, and style. As the results
indicate, there were considerably more errors of terminology and accuracy than uency and style, regardless
of the MT system. Such a result is consistent with MT quality ndings in previous studies (e.g., Heiss &
Soffriti, 2018; Mileto, 2019; Wiesmann, 2019; Wrede et. al., 2019).
More specically, the greatest number of errors pertained to terminology. A number of translations offered by
the NMT systems under study were too literal and ultimately did not encapsulate the meaning of the source
text. This is the case for English translations of Spanish legal system-bound terms, such as names of courts
and legal professionals (e.g., Court of Instruction for Juzgado de Instrucción and Letter for the Administration
of Justice for Letrado de la Administración de Justicia), as well as legal terms highly specic to Spanish
criminal law and procedure (e.g., el investigado being translated as the accused and prisión provisional
comunicada y sin anza being conveyed as provisional prison communicated and without bond). These
ndings corroborate previous results concerning MT and legal terminology (e.g., Killman, 2014; Wiesmann,
2019; Roiss & Zimmermann González, 2020; Roiss, 2021), as current MT engines provide translation options
that are not always contextually appropriate. In some respects, this may be due to the general nature of the
MT systems involved in the study; however, even the domain-specic MT system ran afoul of these errors in
some instances. These terminological errors also manifest across the entirety of the text, insofar as the NMT
systems are not always consistent in their translation of a specic term, instead offering different options for
the same source text unit.
In many cases, accuracy errors in which the options offered by the NMT systems deviate from the source text
meaning correspond primarily to legal false friends (e.g., resolution for resolución and party for partido),
calques (e.g., I give faith for doy fe) and other nonsensical renderings (e.g., indiciously for indiciariamente).
Notwithstanding these terminological issues, our analysis of the NMT-generated output found that NMT
systems were largely successful in generating text that was appropriate with respect to uency and style. In
fact, adequate solutions were provided for frozen language patterns and even for convoluted, obscure legal
phraseological units (see the above discussion on the English translation of dedúzcase testimonio de esta
resolución), which speaks to the utility of MT in the legal domain due to the repetitiveness of legal discourse.
The results of our study reveal how MT systems, which draw on corpora and attempt a phrase-based or even
sentential approach to their output, are successful to varying degrees of accuracy. As far as legal terminology
is concerned, the NMT systems under scrutiny in this study still present major limitations in that they offer
literal translation options which are either ill-suited to the purpose of the translation or fail to render the
specicities of the source legal system that are often implicit in legal terms, and which need to be made explicit
or expanded upon when the translation is intended for informative purposes that do not necessarily mirror
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 86
the context in which the original text was produced (Fuglinskzy & Somssich, 2020). Conversely, in terms of
uency and style, the machine-generated translations analysed in our study presented adequate solutions with
respect to grammar, style, and idiomaticity. As such, MT systems may be useful to translators as a reference
when translating legal texts out of their mother tongue into another language. Since grammar and style are
generally “the two main factors that disclose the non-nativeness of the translator” (Dubĕda, 2021, p. 224),
the use of NMT may clearly help the legal translator meet this challenge and produce a more adequate text
in the target language in terms of uency and style.
The steady increase in the need to translate texts under tight deadlines and subject to budget constraints,
thus with a greater emphasis on speed, is bolstering the use of technology-based translation tools in the legal
domain. Given their ready availability, the use of NMT engines in legal translation merits further research
into the associated benets and shortcomings, taking into account other genres, communicative situations,
and language combinations. Moving forward, additional research needs to address the ways in which legal
translators can take advantage of NMT output and measure the effort required to post-edit NMT-generated
texts in the legal domain. In a similar vein, the impact of pre-editing on legal texts prior to translation using
MT systems would elucidate whether the ndings presented in this article are replicated when a text has been
prepared for this type of translation. The question also arises as to how detection of the most common errors
and their severity, such as those documented in the present case study, could be integrated by the developers
of proprietary MT engines with the aim of improving MT systems, thereby obtaining higher quality output.
While these results are not as easily incorporated into freely available NMT engines such as DeepL, newer
features of these NMT systems, which allow the creation of a glossary to improve the adequacy and cohesion
of terminology, may provide a means by which these case study ndings can support system development. It
would therefore be relevant to re-assess this type of text in the future to determine whether the quality of the
translations generated by these systems improves over time with the incorporation of feedback and additional
data. Hence, the adopted approach to analysis adopted in this case study may serve as a useful means by
which to examine the use of NMT systems in legal settings over time and enable critical reection on their
utility and their shortcomings.
5 References
Abaitua Odriozola, Joseba. (1999). Quince años de traducción automática en España. Perspectives: Studies
in Translatology, 7(2), 221–230. https://doi.org/10.1080/0907676X.1999.9961360
Alcaraz Varó, Enrique, Hughes, Brian, & Gómez González-Jover, Adelina. (2009). El español jurídico. Ariel.
Aldea Sánchez, Paloma, Arróniz Ibáñez de Opacua, Pilar, Ortega Herráez, Juan Miguel, & Plaza Blázquez,
Sonsoles. (2004). Situación actual de la práctica de la traducción y de la interpretación en la
Administración de Justicia. In Ana Luna Alonso & Susana Cruces Colado (Eds.), La traducción en el
ámbito institucional: autonómico, estatal y europeo (pp. 85–126). Universidade de Vigo.
Alonso Jiménez, Elisa, & Calvo Encinas, Elisa. (2015). Developing a blueprint for a technology-mediated
approach to translation studies. Meta: Journal des traducteurs / Meta: Translators’ Journal, 60(1),
135–157. https://doi.org/10.7202/1032403ar
Biel, Łucja. (2017). Quality in institutional EU translation: Parameters, policies and practices. In Tomáš
Svoboda, Łucja Biel, & Krzysztof Łoboda (Eds.), Quality aspects in institutional translation (pp.
31–57). Language Science Press.
Brannan, James. (2017). Identifying written translation in criminal proceedings as a separate right: Scope and
supervision under European law. JoSTrans: Journal of Specialised Translation, 27, 43–57.
Casamayor Maspons, Reynaldo. (2020). La traducción e interpretación en la Administración de Justicia. In
Tanagua Barceló Martínez, Pedro San Ginés Aguilar, & Iván Delgado Pugés (Eds.), Introducción a la
traducción jurídica y jurada (francés–español) (pp. 269–280). Comares.
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 87
Cho, Kyunghyun, van Merrienboer, Bart, Bahdanau, Dzmitry, & Bengio, Yoshua. (2014). On the properties
of neural machine translation: Encoder–decoder approaches. In Dekai Wu, Marine Carpuat, Xavier
Carreras, & Eva Maria Vecchi (Eds.), Eighth Workshop on Syntax, Semantics and Structure in Statistical
Translation (pp. 103–111). Association for Computational Linguistics. https://doi.org/10.3115/v1/W14-
4012
Chromá, Marta. (2016). Traps of legal English as a target language in legal translation. Comparative
Legilinguistics, 26, 71–97. https://doi.org/10.14746/cl.2016.26.04
Dubĕda, Tomáš. (2021). Legal translation into a non-mother tongue. The role of L1 revision. Target, 33(2),
207–227. https://doi.org/10.1075/target.21034.dub
European Commission. (2021, November 3). Machine translation for public administrations: eTranslation.
European Commission.
Feria García, Manuel. (1999). El traductor–intérprete en la Administración de Justicia. In Manuel Feria García
(Ed.), Traducir para la Justicia (pp. 87–108). Comares.
Fuglinszky, Ádám, & Somssich, Réka. (2020). Language-bound terms–Term-bound languages: The difculties
of translating a national civil code into a lingua franca. International Journal for the Semiotics of Law
– Revue internationale de Sémiotique juridique, 33, 749–770. https://doi.org/10.1007/s11196-020-
09704-x
Garofalo, Giovanni. (2009). Géneros discursivos de la justicia penal. Un análisis contrastivo español–italiano
orientado a la traducción. Franco Angeli.
Granados Meroño, Daniel, & Orts Llopis, María Ángeles. (2021). Traducir la corrupción: análisis traductológico
de un auto judicial de la “Operación Púnica”. In Esther Vázquez y del Árbol & María Jesús Ariza
Colmenarejo (Eds.), Aspectos forenses de la traducción y la interpretación jurídica, judicial y policial
(pp. 93–117). Colex.
Heiss, Christine, y Soffritti, Marcelo. (2018). DeepL traduttore e didattica della traduzione dall’italiano in
tedesco. Alcune valutazioni preliminari. In Laurie Anderson, Laura Gavioli, & Federico Zanettin (Eds.),
InTRAlinea. Special Issue: Translation and Interpreting for Language Learners (TAIL). https://www.
intralinea.org/specials/article/2294
Izquierdo Valverde, Isabel. (2016). Nuevo marco legislativo para la interpretación y la traducción judicial en
España tras la Directiva 2010/64/EU. Polissema – Revista de Letras do ISCAP, 16, 17–33. https://doi.
org/10.34630/polissema.v0i16.2957
Jowers, Rebecca. (2016, July 7). Terminology of Spanish judicial decisions. Léxico jurídico español–inglés…
a site for exploring legal terminology.
Keaton, Ashley Rose. (2020). “You have to the right to keep quiet”: Translation inadequacies in Nevada’s
Spanish Miranda warnings. Language and Law / Linguagem e Direito, 7(1–2), 56–76.
Killman, Jeffrey. (2014). Vocabulary accuracy of statistical machine translation in the legal context. In Sharon
O’Brien, Michel Simard, & Lucia Specia (Eds.), Proceedings of the third workshop on post-editing
technology and practice (WPTP–3), The 11th Conference of the Association for Machine Translation
in the Americas (pp. 85–98). Association for Machine Translation in the Americas.
Killman, Jeffrey. (2015). Context as Achilles’ of translation technologies: Major implications for end-users.
Translation and Interpreting Studies, 10(2), 203–222. https://doi.org/10.1075/tis.10.2.03kil
Killman, Jeffrey. (2021). Translation in the shadows of interpreting in US Court systems: Standards, guidelines
and practice. In Fernando Prieto Ramos (Ed.), Institutional translation and interpreting: Assessing
practices and managing for quality (pp. 72–94). Routledge. https://doi.org/10.4324/9780429264894-6
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 88
McAuliffe, Karen. (2013). The limitations of a multilingual legal system. International Journal for the
Semiotics of Law – Revue internationale de Sémiotique juridique, 26, 861–882. https://doi.org/10.1007/
s11196-013-9314-0
Mileto, Fiorenza. (2019). Post-editing and legal translation. H2D, Revista de Humanidades Digitais, 1(1).
https://doi.org/10.21814/h2d.237
Monzó-Nebot, Esther. (2020). Translating genres, creating transgenres: Textual betweens as situation-based
systemic innovations. Estudios de Traducción, 10, 51–67. https://doi.org/10.5209/estr.71633
Nauen, Milana. (2020). Übersetzung von Gerichtsdokumenten. Herausfordernd, richtig und unverständlich (mit
Beispielen aus dem Sprachenpaar Russisch-Deutsch). Babel, 66(2), 254–277. https://doi.org/10.1075/
babel.00147.nau
Neubig, Graham. (2017). Neural machine translation and sequence-to-sequence models: A tutorial. Cornell
University Library.
Ortega Arjonilla, Emilio. (2012). Sobre la traducción judicial: propuesta de conceptualización traductológica.
In Icíar Alonso Araguás, Jesús Baigorri Jalón, & Helen J. L. Campbell (Eds.), Ensayos sobre traducción
jurídica e institucional/Essays on legal and institutional translation (pp. 27–41). Comares.
Ortega Herráez, Juan Miguel, Giambruno, Cynthia, & Hertog, Erik. (2013). Translating for domestic courts
in multicultural regions: Issues and new developments in Europe and the United States of America. In
Anabel Borja Albi & Fernando Prieto Ramos (Eds.), Legal translation in context: Professional issues
and prospects (pp. 89–121). Peter Lang.
Ortega Herráez, Juan Miguel. (2013). Caracterización de la práctica profesional de la traducción en el orden
jurisdiccional penal en España. Puntoycoma, Boletín de los traductores españoles de las instituciones
de la Unión Europea, 133, 10–24.
Orts Llopis, María Ángeles. (2017). El género textual en dos culturas jurídicas: análisis pretraductológico de
las decisiones judiciales en inglés y en español. Revista de Llengua i Dret, Journal of Language and
Law, 67, 220–236. https://doi.org/10.2436/rld.i67.2017.2882
Orts Llopis, María Ángeles. (2018). El género como método de estudio y enseñanza en la traducción de textos
especializados en inglés-español. El género y la traducción jurídica. In Ana María Rojo López (Ed.),
La investigación en traducción: aspectos metodológicos y aplicaciones didácticas (pp. 125–157).
Anthropos.
Oxford University Press. (n.d.). Detention. In A dictionary of law.
Oxford University Press. (n.d.). Kidnapping. In A dictionary of law.
Oxford University Press. (n.d.). Remand (in custody) order. In A dictionary of law.
Papineni, Kishore, Roukos, Salim, Ward, Todd, & Zhu, Wei-Jing. (2002). Bleu: A method for automatic
evaluation of machine translation. In Isabelle Pierre, Eugene Charniak, & Dekang Lin (Eds.),
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (pp. 311–318).
Association for Computational Linguistics. https://doi.org/10.3115/1073083.1073135
Pasteur, Olivier. (2013). Technology at the service of specialized translators at international organizations. In
Anabel Borja Albi & Fernando Prieto Ramos (Eds.), Legal translation in context:. Professional issues
and prospects (pp. 283–297). Peter Lang.
Prieto Ramos, Fernando. (2014). Parameters for problem-solving in legal translation: Implications for legal
lexicography and institutional terminology management. In Le Cheng, Kui King Sin, & Anne Wagner
(Eds.), The Ashgate handbook of legal translation (pp. 121–134). Ashgate.
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 89
Real Academia Española. (n.d.). Auto de prisión provisional. In Diccionario panhispánico del español jurídico
(DPEJ).
Real Academia Española. (n.d.). Juez. In Diccionario panhispánico del español jurídico (DPEJ).
Roiss, Silvia, & Zimmermann González, Petra. (2020). DeepL y su potencial para el desarrollo de la capacidad
de análisis crítico en la clase de traducción inversa. Hermeneus, 22, 363–382. https://doi.org/10.24197/
her.22.2020.363-382
Roiss, Silvia. (2021). Y las máquinas rompieron a traducir… Consideraciones didácticas en relación con la
traducción automática de referencias culturales en el ámbito jurídico. Trans. Revista de Traductología,
25, 491–505. https://doi.org/10.24310/TRANS.2021.v1i25.11978
Scarpa, Federica, Peruzzo, Katia, & Pontrandolfo, Gianluca. (2014). Methodological, terminological and
phraseological challenges in the translation into English of the Italian Code of Criminal Procedure.
In Mitja Gialuz, Luca Lupária, & Federica Scarpa (Eds.), The Italian Code of Criminal Procedure:
Critical essays and English translation (pp. 53–80). Wolters Kluwer.
Sommerlad, Joe. (2021, March 24). Google Translate: how does the multilingual interpreter actually work?
Independent.
Spanish General Council of the Judiciary. (n.d.). Activity of judicial bodies – Actividad de los traductores.
Poder Judicial España.
TAUS. (2021). QT21 project: Harmonized error typology. TAUS – The Language Data Network.
USLegal. (n.d.). Illegal detention. In The USLegal dictionary.
Vigier-Moreno, Francisco Javier. (2020). Análisis pretraslativo multinivel del género jurídico escrito de
acusación. Elementos relevantes para su traducción al inglés. Estudios de Traducción, 10, 35–49.
https://doi.org/10.5209/estr.68053
Vigier-Moreno, Francisco Javier, & Pérez-Macías, Lorena. (2020). ¿Es ecaz la posedición para la traducción
inversa de textos jurídicos? Una experiencia con estudiantes de posgrado. In David Cobos-Sanchiz,
Eloy López-Meneses, Laura Molina-García, Alicia Jaén-Martínez, & Antonio Hilario Martín-Padilla
(Eds.), Claves para la innovación pedagógica ante los nuevos retos: respuestas en la vanguardia de
la práctica educativa (pp. 2499–2508). Octaedro.
Wiesmann, Ewa. (2019). Machine translation in the eld of law: A study of the translation of Italian legal
texts into German. Comparative Legilinguistics, 37, 117–153. https://doi.org/10.14746/cl.2019.37.4
Wrede, Oľga, Munková, Daša, & Welnitzová, Katarína. (2020). Effektivität des Post-Editings maschineller
Übersetzung: eine Fallstudie zur Übersetzung von Rechtstexten aus dem Slovakischen ins Deutsche.
Lingua et Vita, 17, 117–126.
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 90
6 Appendix (Spanish remand order)
JUZGADO DE INSTRUCCIÓN N°3
C/Isla Mallorca, s/n
Córdoba
Teléfono: 957745084
Fax.: 957002347
Procedimiento: DILIGENCIAS PREVIAS
N° Procedimiento: 000456/2018
AUTO
En Córdoba, a 17 de junio de 2018.
HECHOS
PRIMERO.- De lo actuado resulta indiciariamente acreditado a los efectos de esta Resolución que durante la
noche de los días 13 a 14 de junio de 2018, [name redacted], de nacionalidad [nationality redacted], llevó a
Juan Fernández Gómez hasta la zona denominada Las Cañadas, en el término municipal de Córdoba, donde le
golpeó con un madero en la cabeza causándole una fractura de cráneo que acabaría produciéndole la muerte,
quitándole acto seguido sus pertenencias y utilizando el teléfono móvil de la víctima para exigir un rescate
a su familia.
SEGUNDO.- En la audiencia prevista en el artículo 505 de la Ley de Enjuiciamiento Criminal el Ministerio
Fiscal ha solicitado la prisión provisional.
RAZONAMIENTOS JURÍDICOS
PRIMERO.- Conforme a lo dispuesto en los artículos 503 y 505 de la Ley de Enjuiciamiento Criminal para
que proceda la prisión provisional se requiere:
1.1.1) Que conste en la causa la existencia de un hecho que tenga caracteres de delito.
1.1.2) Que este tenga señalada pena igual o superior a dos años de prisión, o que, teniéndola inferior
se considere procedente en atención a los antecedentes penales del inculpado.
1.1.3) Que existan motivos bastantes para estimar responsable criminalmente a la persona contra la
que se haya de dictar el auto de prisión.
1.1.4) Que se haya celebrado la audiencia prevista en el artículo 505 de la Ley de Enjuiciamiento
Criminal, con asistencia del investigado y del Ministerio Fiscal.
1.1.5) Que la prisión haya sido solicitada por el Ministerio Fiscal o parte acusadora.
Mediante la prisión provisional se persigue algunos de los siguientes nes:
1.2.1) Asegurar la presencia del investigado en el proceso cuando pueda inferirse racionalmente un
riesgo de fuga. Para valorar la existencia de este peligro se atenderá conjuntamente a la naturaleza del
Francisco J. Vigier-Moreno & Lorena Pérez-Macías
Assessing neural machine translation of court documents: A case study on the translation...
Revista de Llengua i Dret, Journal of Language and Law, 78, 2022 91
hecho, a la gravedad de la pena que pudiera imponerse al investigado, a la situación familiar, laboral
y económica de este, así como a la inminencia de la celebración del juicio oral.
1.2.2) Evitar la ocultación, alteración o destrucción de las fuentes de prueba relevantes para el
enjuiciamiento en los casos en que exista un peligro fundado y concreto. Para valorar la existencia
de este peligro se atenderá a la capacidad del investigado para acceder por sí o a través de terceros a
las fuentes de prueba o para inuir sobre otros investigados o encausados, testigos o peritos o quienes
pudieran serlo.
1.2.3) Evitar que el investigado pueda actuar contra bienes jurídicos de la víctima, especialmente
cuando esta sea alguna de las personas a las que se reere el artículo 173.2 del Código Penal.
1.2.4) Evitar el riesgo de que el investigado cometa otros hechos delictivos. Para valorar la existencia
de este riesgo se atenderá a las circunstancias del hecho, así como a la gravedad de los delitos que
se pudieran cometer.
SEGUNDO.- En el presente caso concurren todos los requisitos mencionados por cuanto del relato de Hechos
expuesto en los de esta Resolución y de lo actuado hasta ahora en la causa se desprende la existencia de
delitos de homicidio, detención ilegal y robo con violencia a los que el Código Penal en sus artículos 138
y 139, 183 y siguientes y 237 y 242 señala pena superior a dos años, existen en la causa méritos bastantes
para estimar responsable criminalmente de dicho delito a [name redacted] (en concreto la ocupación de los
efectos sustraídos, el registro del lugar que habitaba, las llamadas exigiendo un rescate y lo declarado por el
propio investigado), se ha celebrado la audiencia que exige la Ley y por último la prisión provisional ha sido
solicitada por el Ministerio Fiscal.
TERCERO,- Por lo expuesto procede decretar la prisión provisional comunicada y sin anza por esta causa de
[name redacted], teniendo en cuenta la gravedad de los hechos, los bienes personales en juego, la necesidad de
asegurar la instrucción así como las elevadas posibilidades de eludir la acción de la justicia ante la gravedad
de las penas que en su día pudieran imponerse y la falta de un domicilio able.
PARTE DISPOSITIVA
Se decreta por esta causa la PRISIÓN PROVISIONAL COMUNICADA Y SIN FIANZA DE [name redacted].
Para llevarla a efecto líbrese mandamiento al Director del Centro Penitenciario que en Resolución aparte se
determinará y ocio a la fuerza policial actuante.
Dedúzcase testimonio de esta Resolución para formar la correspondiente pieza de situación personal.
Notifíquese este Auto al Ministerio Fiscal, al investigado y a las demás partes personadas haciéndoles saber
que no es rme y que contra el mismo cabe recurso de reforma por escrito presentado en este juzgado en el
plazo de tres días y subsidiariamente recurso de apelación para ante la Audiencia Provincial de Córdoba que
también puede interponerse directamente en el plazo de cinco días.
Así lo acuerda, manda y rma D. FERNANDO DE MERA RODRÍGUEZ, Magistrado-Juez del Juzgado de
Instrucción número 3 de Córdoba y su partido.- Doy fe.
DILIGENCIA.- Seguidamente se cumple lo acordado, doy fe.
LETRADO DE LA ADMINISTRACIÓN DE JUSTICIA
