Natural language processing resources - Data

English-Hindi Dictionary

Machine Learning resources - tools

Graclus - graph clustering

Natural language processing resources

IR

• http://www.sigir.org/resources.html

More pointers

• http://nlp.stanford.edu/IR-book/information-retrieval.html
• http://terpconnect.umd.edu/~oard/teaching/796/spring04/research.html
• http://www.lt-world.org/

Natural language processing resources - Tools

IR

• Indri
• Lucene

General text analysis

• TMG
  
• GATE
• Lingpipe
• UIMA
• GIZA++
• OpenNLP
• Gensim 
• Sketch Engine
• BootCaT

More pointers

• http://www.searchtools.com/tools/tools-opensource.html

P.S.-- I will keep adding links as and when I find them.

ACL HLT workshop

WORKSHOP ON BUILDING AND USING COMPARABLE CORPORA

   Comparable Corpora and the Web

   Co-located with ACL-HLT 2011
   Portland, Oregon
   24 June 2011

   DEADLINE FOR PAPERS: 1st April 2011
   http://www.limsi.fr/~pz/bucc2011-comparable-corpora/
   Submission: https://www.softconf.com/acl2011/comparable/

---------------------------------------------------------

Btw, here is a definition for comparable corpora from the CFP : "documents in one or several languages that are comparable in content and form in various degrees and dimensions".

Paper notes - Dual Decomposition for Parsing with Non-Projective Head Automata

Koo, Rush, Collins, Jaakkola, Sontag. EMNLP 2010.

Problem: Non-projective head automata (I dont know this) parsing - building a parse tree for a sentence, using annotated data (sentences and their parse trees).

Contribution: A dual decomposition formulation of the problem. Leads to efficient algorithms based on minimum spanning tree (MST) algorithms and dynamic programming.

TODO : Got stuck in understanding the basics of the optimization and algorithms involved. Will get back to this later.

Some of the things to look up -

• Head automata; parsing
  
• Dual decomposition, Lagrangian relaxation - in combinatorial optimization
  
• LP/ILP solvers
• MST based optimization
• NP-hard/complete - practice judging hardness of problems
  

Paper notes - Beyong NomBank: A Study of Implicit Arguments for Nominal Predicates

Gerber and Chai. ACL 2010

Problem: (Unclear to me) Identify arguments for the predicates in a sentence, even when they are implicit (having been mentioned earlier, in preceding sentences).

Contribution:

• They have introduced a new problem. Earlier only explicit arguments were considered and their recognition was measured.
• They have annotated data for the new problem and made it available as a resource for the community. Also, they established a baseline for future work.
  

Learning points

• [IMP] Start introduction with a motivating example, that immediately elucidates the problem cleanly, and clarifies differences with existing problems/methods.
• My ShoBha work also introduces a new problem; this might be a good model paper to emulate for my draft.
• Introduction is short, crisp and has 3 parts - motivating example, "what is the problem" and "what is our contribution".
• A whole section devoted to the manual efforts of annotation and resource building. We should do this for our work on the Wikipedia data.
• Insightful comments and statistics about the data (especially those that are pertinent to the problem on hand).
• Wherever annotation as used, Cohen's kappa coefficient (Cohen, 1960) was mentioned.
• A careful design of features and detailed analysis using - (1) Floating forward feature selection (2) Grid search (3) Feature classes
  
• Some packages used - (1) LibLinear's logistic regression solver (for analyzing feature classes) (2) OpenNLP coreference identification
• Choose a reasonable baseline (possible based on some heuristic)
• Look at old papers for the evaluation methodology that is prevalent for this/similar problem/s.
• Dice coefficient for measuring performance. All results were reported with statistical significance, using two-tailed bootstrap method (Efron and Tibshirani, 1993).
• Ablation Study (in the 'Discussion' section) w.r.t. the features.
• Error analysis - an example when it failed, and why did it fail.
• Success analysis - an example when it worked, and why.