RIBES: Rank-based Intuitive Bilingual Evaluation Measure

What's RIBES?

RIBES is an automatic evaluation metric for machine translation, developed in NTT Communication Science Labs.
This website distributes its implementation in Python.

Download

The program is distributed based on GNU General Public License (ver.2)．
You have to confirm the agreement with the license terms before download.
Click to Download : Latest version 1.02.3 (2012/2/23) -- fixed a problem in output
Click to Download : Version 1.01 (2011/8/10) -- bug fix on empty input
* Requires Python 3.1 or later．

How to Use

$ python3 RIBES.py -r REFERENCE_TRANSLATION EVALUATING_TRANSLATION

Also you can see other options by the following:

$ python3 RIBES.py -h

File Format

Reference and evaluating files are plain text files, not SGML or XML.
Reference and evaluating files must have the same number of lines.
Each line must be tokenized including punctuations (The program does not change tokenization).

References

Hideki Isozaki, Tsutomu Hirao, Kevin Duh, Katsuhito Sudoh, Hajime Tsukada
Automatic Evaluation of Translation Quality for Distant Language Pairs (PDF)
Conference on Empirical Methods on Natural Language Processing (EMNLP), Oct. 2010.

Contact: (the name of our evaluation measure, lowercased)@lab.ntt.co.jp