Readings on Morphology & MT

MT Reading Group, April 19, 2006

Goldwater & McClosky (2005)

(01) Table: Morphological correspondences

	Czech	English
a.	plural past tense	plural past tense
b.	person negation genitive case instrumental case	pronouns not of by or with
c.	gender	none
d.	other case markings	syntax

Introduction

• Estimating word-to-word alignments for the translation model is hindered by sparse data.
• Much of the morphological variation in Czech is not reflected in English.
• Final improvement: BLEU .270 → .333

Methods

3.1 Lemmas

• Replace each word form with its associated lemma (i.e. root).
  1. Lemmatize only certain POS.
  2. Lemmatize all POS except pronouns.
  3. Lemmatize only rare words.
  4. Truncate to n characters.
• Reduces data sparseness, but may lose useful information.
• Expected to help for (01c,d).

Morpheme Tokens (3.2 Pseudowords)

• Each token encodes a single morphological tag.
• Reduces data sparseness, although not as much as lemmatization.
• Retains correspondences with English function words.
• Expected to help for (01b).

3.3 Modified Lemmas

• Lemmatize, then concatenate tags of interest.
• Eliminates some variation among wordforms.
• Slightly reduces data sparseness.
• Expected to help for (01a).

Feature Vectors (3.4 Morphemes)

• Replace fj → fj0 … fjK.  Calculate:

  (02)	P(fj|ei) = ∏k P(fjk|ei)

  How are probabilities affected by null fjk?
• Estimate using variant of EM algorithm for alignment.
• Reduces data sparseness, but retains potentially useful information.
• Allows backoff to morphological tags.

(03) Figure: Transformations

a.	Original:	Pro nĕkoho by její provedení mĕlo smysl .
b.	Lemmas:	pro nĕkdo být jeho provedení mít smysl .
c.	Morpheme Tokens:	pro nĕkdo být PER_3 jeho provedení mít PER_X smysl .
d.	Modified Lemmas:	pro nĕkdo být+PER_3 jeho provedení mít+PER_X smysl .
e.	Vector:	(pro) (nĕkdo) (být PER_3) (jeho) (provedení) (mít PER_X) (smysl) (.)

4.5 Combined Model

Based on the results of the above experiments, they created a combined model using tokens for person and negation morphemes, and the modified lemma method for number and tense.

Data

• PCEDT corpus
• 250 sentences each dev/test, translated Czech→English by 5 translators for BLEU scores
• ~21,000 sentences translated to Czech and morphologically annotated
• ~50,000 English sentences to train language model

Roughly same number of rare lemmas in English and Czech, but roughly twice as many rare inflected wordforms in Czech.

(05) Table: Occurrences of each class of tag

	Tag Class	Count
a.	PER	49700
b.	TEN	47744
c.	past	22544
d.	present	20291
e.	future	1707
f.	other	3202
g.	NUM	151646
h.	CASE	151646
i.	NEG	3326

(04) Figure: Number of items y with token count of x.

Results

Experiments were with a word-to-word (not phrase-based) translation system.

BLEU score difference of .009 is significant (p < .05).

(06) Table: BLEU scores for baseline & lemmatization

		Dev	Test
a.	word-to-word	.311	.270
b.	truncate (k=6)	.353	.283
c.	lemmatize all	.355	.299
d.	except Pro	.350	–
e.	except Pro, V, N	.346	–
f.	n < 50	.370	.306

(07) Table: BLEU scores (Dev set) with different methods & classes of tags

		Tokens	Mod-Lem	Vectors
a.	PER	.365	.356	.356
b.	TEN	.365	.361	.364
c.	PER,TEN	.355	.362	.355
d.	NUM	.354	.367	.361
e.	CASE	.353	.340	.337
f.	NEG	.357	.356	.353

(08)

	Dev	Test
combined model	.390	.333

The best lemmatization results occurred with lemmatizing low-frequency words (06f).  Truncation (06b) performed significantly better than baseline (06a), but significantly worse than lemmatization.

Scores on other experiments (07) are reported on the Dev set only.  None scored better than lemmatization (!).

Adding person-agreement tokens (PER) helped, and examination confirmed that they did, indeed, align with English pronouns.  Inexplicably, tense-marking tokens (TEN) helped just as much, but using both PER and TEN canceled the benefits of either.

For modified lemmas, number (NUM) and tense tags helped, which makes sense since these are also marked in English.  Case tags hurt, which also makes sense, since they increase data sparseness without providing any information corresponding with English.

Feature vectors did not perform significantly better than morpheme tokens in any experiment (!).

The combined model performed best of all, successfully combining the advantages of each method.

Conclusions

1. Morphological analysis does help, through the reduction of data sparseness.
2. The greatest improvement comes from changing the source text to reflect the morphological characteristics of the target text.

Schrader (2006)

Introduction

German→English alignment is hindered by long German compounds aligning with English phrases, and a large proportion of hapax legomena.

This system aligns based on word length and POS patterns, without relying on frequency counts.

Nießen & Ney (2001) and Tschorn & Lüdeling (2003) split German compounds into components.  This only works if the meaning of the compound is compositional (09).

(09)	Personen-stand	'marital status'
	personal-status

This criticism suggests word-based translation.

Data

• Europarl corpus
• both sides POS tagged
• manually corrected alignments of ~100,000 German→English tokens

(10) Table: Corpus characteristics

Language	Tokens	Types	Hapax Legomena	Other Rare	Frequent
English	29,077,024	101,967	39,200 (38%)	35,608 (35%)	27,159 (27%)
German	27,643,792	286,330	140,826 (49%)	98,126 (34%)	47,378 (17%)

Of 512 German hapax legomena examined:

• 68.95% are noun compounds
• 68% align with English multi-word expressions, mostly noun sequences preceded by an Adj or followed by a PP
• correspondance of German compound length to English multi-word phrase length is high (11)

(11) Table: German compound length vs. English phrase length

	German compound length
English	1	2	3	4	>4
1 word	59	2	1	0	3
2 words	30	119	56	15	10
3 words	2	20	15	8	10
4 words	0	1	0	0	2

The English phrase length is often 1 more due to PP (12).

(12)	Kongresh-vorlage
	submission to Congress

Compound Alignment Strategy

• German token is considered a compound if it is tagged as a noun and is at least 12 characters long
• English candidate phrases are generated from noun sequences with optional preceding Adj and/or following PP
• length ratio is compared:
  0  <  1 – (German compound length / English phrase length)
• qualifying candidate is added to bilingual dictionary

What happens if there are multiple qualifying candidates?

Results

• 1600 additional entries in bilingual dictionary
  additional to what?
• 236 of 353 known hapax compounds were translated
  • 11 unidentified compounds did not meet length threshold
  • 248 of 353 after improvements to nominal recognizing heuristics
• of hapaxes added to dictionary, 47% were translated correctly
  • 70% after improvements

Future Work

• integrate into SMT
• use morphological analysis instead of length/POS heuristics

Koehn & Knight (2003)

• German→English 1→many
• learn rules for splitting German compounds
• cover each German compound with known words and fillers
1. aktionsplan
2. aktions-plan
3. aktion-s-plan
4. akt-ion-s-plan
• select best covering based on various heuristics
1. eager – split into most possible chunks
2. frequency based – metric
3. parallel text – find best mapping to English translation
4. POS – limit covering words to content words (not P,DET)
• 99.1% accuracy using parallel text & POS
• .039 improvement to BLEU using eager

(13) Table: Evaluation of compound-splitting methods

	Metrics			BLEU
	prec	recall	acc	WBSMT	PBSMT
none	–	0.0%	94.2%	.291	.305
eager	24.8%	73.3%	87.1%	.222	.344
frequency based	57.4%	86.6%	95.7%	.317	.342
parallel	83.3%	89.1%	98.6%	.294	.330
parallel & POS	93.8%	90.1%	99.1%	.306	.326

Lee (2004)

• Arabic→English 1→many
• analyze Arabic into prefix(es)-stem-suffix(es), distinct tokens
• align Arabic morphemes to English
• unaligned morphemes are either deleted or merged with stem based on alignment frequencies
• Model 1, doubles BLEU score
• PBSMT, equivalent to an order of magnitude more data

(14) Table: BLEU-score evaluation of delete/merge

	Model 1		PBSMT
# sentences	base	morph	base	morph
3.5K	.10	.25	.17	.24
35K	.14	.29	.24	.29
350K	.18	.31	.32	.36
3.3M	.18	.32	.36	.39

Habash & Rambow (2005)

• Arabic→English
• possible morphological classes
• English: 50
• Arabic: 333,000 theoretically, 2200 attested

Nießen & Ney (2001,2004)

• German→English
• hierarchical lexicon model
• f0..k is considered special case of base form f0..k-1
• align at all levels
• syntactic restructuring
• reorder questions
• detach verb prefixes
• augmented dictionary
• include separate entries for homonymous forms with different morphology, e.g. moose(sg.) vs. moose(pl.)
• detect multiword phrases based on POS sequence
• improvements comparable to an order of magnitude of data

(15) Table: Evaluation of methods (lower is better)

#sent	Method	1-BLEU	mWER	SSER	ISER
0	baseline	76.7	53.6	60.4	29.8
	restructure	70.9	50.2	57.8	30.0
	+ all	67.4	48.0	52.8	24.1
5K	baseline	52.6	38.0	37.3	17.4
	restructure	47.9	34.7	33.6	15.2
	+ all	47.1	33.9	31.8	13.7
58K	baseline	46.3	34.1	30.2	14.1
	restructure	43.7	32.5	26.6	12.8
	+ all	42.9	31.8	26.3	11.8

General Discussion

1. data sparseness is a real problem, even with large corpora
• 50% hapax legomena in German
• 2x as many rare wordforms in Czech vs. English
2. morphological analysis helps, up to an order of magnitude more data
3. most experiments so far involve techniques highly customized for specific language pair
4. most significant improvements are from changes which take the structure of the target language into consideration
5. PBSMT overcomes overeager tokenization, may counteract (D)
6. improvements are expected to correlate with inverse size of training corpus
7. Future: factored translation models

References