Brill tagger

The Brill tagger is an inductive method for part-of-speech tagging. It was described and invented by Eric Brill in his 1995 PhD thesis. It can be summarized as an "error-driven transformation-based tagger". It is

• a form of supervised learning, which aims to minimize error
• transformation-based in the sense that a tag is assigned to each word and changed using a set of predefined rules. Note: If the word is known, it first assigns the most frequent tag, or if the word is unknown, it naively assigns the tag "noun" to it. Applying over and over these rules, changing the incorrect tags, a quite high accuracy is achieved. This approach ensures that valuable information such as morphosyntact construct of words are employed in an automatic tagging process.

Algorithm

The algorithm starts with initialization, which is the assignment of tags based on their probability for each word (for example, "dog" is more often a noun than a verb). Then "patches" are determined via rules that correct (probable) tagging errors made in the initialization phase:^[1]

• Initialization:
  • Known words (in vocabulary): assigning the most frequent tag associated to a form of the word
  • Unknown word

Rules and processing

The input text is first tokenized, or broken into words. Typically in natural language processing, contractions such as "'s", "n't", and the like are considered separate word tokens, as are punctuation marks.

A dictionary and some morphological rules then provide an initial tag for each word token. For example, a simple lookup would reveal that "dog" may be a noun or a verb (the most frequent tag is simply chosen), while an unknown word will be assigned some tag(s) based on capitalization, various prefix or suffix strings, etc. (such morphological analyses, which Brill calls Lexical Rules, may vary between implementations).

After all word tokens have (provisional) tags, contextual rules apply iteratively, to correct the tags by examining small amounts of context. This is where the Brill method differs from other part of speech tagging methods such as those using Hidden Markov Models. Rules are reapplied repeatedly, until a threshold is reached, or no more rules can apply.

Brill rules are of the general form:

   tag1 → tag2 IF Condition

where the Condition tests the preceding and/or following word tokens, or their tags (the notation for such rules differs between implementations). For example, in Brill's notation:

   IN NN WDPREVTAG DT while

would change the tag of a word from IN (preposition) to NN (common noun), if the preceding word's tag is DT (determiner) and the word itself is "while". This covers cases like "all the while" or "in a while", where "while" should be tagged as a noun rather than its more common use as a preposition (many rules are more general).

Rules should only operate if the tag being changed is also known to be permissible, for the word in question or in principle (for example, most any adjective in English can also be used as a noun).

Rules of this kind can be implemented by simple Finite-state machines. See Part of speech tagging for more general information including descriptions of the Penn Treebank and other sets of tags.

Typical Brill taggers use a few hundred rules, which may be developed by linguistic intuition or by machine learning on a pre-tagged corpus.

Code

Brill's code pages at Johns Hopkins University are no longer on the web. A mirror of the Brill tagger at its latest version is available at Plymouth Tech, here. The software uses the MIT License.

References

1. ↑ Eric Brill. 1992. A simple rule-based part of speech tagger. In Proceedings of the third conference on Applied natural language processing (ANLC '92). Association for Computational Linguistics, Stroudsburg, PA, USA, 152-155. doi:10.3115/974499.974526

External links

• Brill tagger trained for Dutch (online and offline version)
• Brill tagger trained for New Norwegian
• Brill tagger trained for Danish
• Brill tagger trained for English