Comparative Quality Estimation is the automatic process of analyzing two or more translations produced by machine translation systems and expressing a judgment about their comparison. We approach the problem from a supervised machine learning perspective, with the aim to learn from human preferences. As a result, we create the ranking mechanism, a pipeline that includes the necessary tasks for ordering several MT outputs of a given source sentence in terms of relative quality.
Quality Estimation models are trained to statistically associate the judgments with some qualitative features, with additional focus on the ones with a grammatical background. We learn the model with binary classifiers after decomposing the ranking into pairwise decisions. The predictions correlate to the ones given by humans and can be successfully compared with state-of-the-art metrics and other known ranking methods. We also apply this method for a hybrid MT system combination with positive results.
Eleftherios is a Researcher in the Language Technology Lab of the German Research Center for Artificial Intelligence and he is about to defend his PhD under the supervision of Prof. Hans Uszkoreit. He has worked for Quality Estimation and Machine Translation for several EU research and development projects and he has recently interned in Google Translate.