Paper notes - Beyong NomBank: A Study of Implicit Arguments for Nominal Predicates

Gerber and Chai. ACL 2010

Problem: (Unclear to me) Identify arguments for the predicates in a sentence, even when they are implicit (having been mentioned earlier, in preceding sentences).

Contribution:

• They have introduced a new problem. Earlier only explicit arguments were considered and their recognition was measured.
• They have annotated data for the new problem and made it available as a resource for the community. Also, they established a baseline for future work.
  

Learning points

• [IMP] Start introduction with a motivating example, that immediately elucidates the problem cleanly, and clarifies differences with existing problems/methods.
• My ShoBha work also introduces a new problem; this might be a good model paper to emulate for my draft.
• Introduction is short, crisp and has 3 parts - motivating example, "what is the problem" and "what is our contribution".
• A whole section devoted to the manual efforts of annotation and resource building. We should do this for our work on the Wikipedia data.
• Insightful comments and statistics about the data (especially those that are pertinent to the problem on hand).
• Wherever annotation as used, Cohen's kappa coefficient (Cohen, 1960) was mentioned.
• A careful design of features and detailed analysis using - (1) Floating forward feature selection (2) Grid search (3) Feature classes
  
• Some packages used - (1) LibLinear's logistic regression solver (for analyzing feature classes) (2) OpenNLP coreference identification
• Choose a reasonable baseline (possible based on some heuristic)
• Look at old papers for the evaluation methodology that is prevalent for this/similar problem/s.
• Dice coefficient for measuring performance. All results were reported with statistical significance, using two-tailed bootstrap method (Efron and Tibshirani, 1993).
• Ablation Study (in the 'Discussion' section) w.r.t. the features.
• Error analysis - an example when it failed, and why did it fail.
• Success analysis - an example when it worked, and why.