Thursday, December 9, 2010

Paper notes - Beyong NomBank: A Study of Implicit Arguments for Nominal Predicates

Gerber and Chai. ACL 2010

Problem: (Unclear to me) Identify arguments for the predicates in a sentence, even when they are implicit (having been mentioned earlier, in preceding sentences).

  • They have introduced a new problem. Earlier only explicit arguments were considered and their recognition was measured.
  • They have annotated data for the new problem and made it available as a resource for the community. Also, they established a baseline for future work.
Learning points
  • [IMP] Start introduction with a motivating example, that immediately elucidates the problem cleanly, and clarifies differences with existing problems/methods.
  • My ShoBha work also introduces a new problem; this might be a good model paper to emulate for my draft.
  • Introduction is short, crisp and has 3 parts - motivating example, "what is the problem" and "what is our contribution".
  • A whole section devoted to the manual efforts of annotation and resource building. We should do this for our work on the Wikipedia data.
  • Insightful comments and statistics about the data (especially those that are pertinent to the problem on hand).
  • Wherever annotation as used, Cohen's kappa coefficient (Cohen, 1960) was mentioned.
  • A careful design of features and detailed analysis using - (1) Floating forward feature selection (2) Grid search (3) Feature classes
  • Some packages used - (1) LibLinear's logistic regression solver (for analyzing feature classes) (2) OpenNLP coreference identification
  • Choose a reasonable baseline (possible based on some heuristic)
  • Look at old papers for the evaluation methodology that is prevalent for this/similar problem/s.
  • Dice coefficient for measuring performance. All results were reported with statistical significance, using two-tailed bootstrap method (Efron and Tibshirani, 1993).
  • Ablation Study (in the 'Discussion' section) w.r.t. the features.
  • Error analysis - an example when it failed, and why did it fail.
  • Success analysis - an example when it worked, and why.

No comments:

Post a Comment