Vocabulary in English

I've been trying to find resources pertaining to the frequency of words in English - surely there must be some kind of graded scale of "commonness" or something? But so far I can't find anything that organized.

Instead, I've got two interesting links here:

• English for Academic Purposes
• 1000 most common SAT words

If we consider each of those and their ilk to be "general vocabulary words", then we'll have that much more luck in identifying technical content in a given document.

TaaS

Terminology as a Service, as envisaged by the Europeans.

Terminology resources

A bit of a linkdump here, first:

• WordNet is now at 3.0 on Unix, still 2.1 on Windows. The database from Linux is probably more useful. Interestingly, it's also available in Prolog. The licensing is pretty open these days. I don't think it used to be. That's welcome news.
• Here's something called CoreLex.
• A good overview of the OLIF format.

I think I could do worse for termbase storage in Perl than simply a database schema that mirrors OLIF (at least partly). That could be part of a general OLIF-handling set of modules. OLIF is attractive because it's model-agnostic in terms of how terms are conceptualized, so an OLIF-based module should be able to do something reasonable with essentially any terminological source.

Lingua::OLIF?

TreeTagger

The Perl module Lingua::TreeTagger provides an interface to the TreeTagger program (Win installation here).  The only problem with TreeTagger is that it's got a commercial-license requirement. That, and the approach isn't good for Hungarian - but you can't have everything.  This would probably be the best possible intermediate structure for frame-based translation, which I still think should be a valid approach.

Word lists

I really need some kind of principled way to keep track of word lists and terminology.  Ideally this would be a full-blown terminology management system with an online component and everything, but it would also be a word list source.

Here are some good places to start with word lists:

• http://wordlist.sourceforge.net/
• http://www.puzzlers.org/dokuwiki/doku.php?id=solving:wordlists:start

File::TMX

I just started scratching the surface for TMX files.  I'm going to end up with some generalized useful tools for XML files.

Terminology source: EMA

The European Medicines Agency publishes patient information about European-approved (not nationally approved) drugs in all the European languages.  This would be a useful corpus for terminological (and syntactic) analysis.