Main Page

From Ritewiki

Jump to: navigation, search
The latest information is available in the NTCIR-10 RITE-2 website or in Twitter @ntcir10_rite

Overview

RITE is a generic benchmark task that addresses major text understanding needs in various NLP/Information Access research areas. We will evaluate systems which automatically detect entailment, paraphrase, and contradiction in texts written in Japanese, Simplified Chinese, or Traditional Chinese. In BC subtask, an input text pair looks as follows:

t1: Yasunari Kawabata won the Nobel Prize in Literature for his novel "Snow Country"
t2: Yasunari Kawabata is the writer of "Snow Country"

A system is given a credit for this pair if it returns the same label as human assessors'; in this specific case, "Y" (meaning "Yes", a human reading the premise t1 would infer that the hypothesis t2 is most likely true).


NTCIR-9 RITE Subtasks


Who should participate?

RITE is a generic task that addresses major Text Understanding needs in various research areas e.g. Information Retrieval, Question Answering, Text Summarization, Opinion Analysis. Researchers in Natural Language Processing, Machine Learning community are also very welcome. Since RITE is a friendly evaluation effort, do not hesitate to try ambitious/exploratory approaches that do not necessarily work. A uniqueness in your method would also be a big contribution to the community. Don't worry about the lack of language resource/tool to participate. We will build the Resource Pool on this wiki, so that useful resources can be shared among participants, and they can be used to build a system for the formal evaluation.


Menu


Current Status

Schedule

Date Event
2011-01~03 Training data development
2011-05- Dry run
2011-04-06 Test data development
2011-09 19-25 Formal run for BC & MC subtask
2011-09-27 Evaluation results return for BC & MC
2011-10-01 Participant paper draft due
2011-10-04 Training data release for EXAM
2011-10 12-19 Formal run for EXAM & RITE4QA
2011-11-04 Participant paper due (firm deadline)
2011-12 NTCIR-9 Workshop Meeting

Detailed schedule is available here.

Mailing list

If you are interested in the task, join our mailing list by sending an empty email to ntc9-rite-list-subscribe [at] cs.cmu.edu

Organizers

Language Organizers
Japanese Hideki Shima (Carnegie Mellon University)
Teruko Mitamura (Carnegie Mellon University)
Hiroshi Kanayama (IBM Research - Tokyo)
Koichi Takeda (IBM Research - Tokyo)
Yusuke Miyao (National Institute of Informatics)
Simplified Chinese Shuming Shi (Microsoft Research Asia)
Traditional Chinese Chuan-Jie Lin (National Taiwan Ocean University)
Cheng-Wei Lee (Academia Sinica)

Email ntc9-rite-organizers [at] cs.cmu.edu if you have any questions.

Links


Personal tools
Namespaces
Variants
Actions
Menu (English)
メニュー (日本語)
目次 (繁體中文)
Navigation
Toolbox