Software Project: Linguistic Interpretibility for Neural Models of Language

Thanks to those of you who have enrolled in this software project! You should all have received an email from me by now, confirming your participation in this course.

The first milestone for course participation is the project proposal. This should be 3 pages in ACL format. I have extended the due date from May 1 to Monday May 5. Your proposal should clearly identify:

• The linguistic feature/s you want to investigate, with citation/s to relevant literature. 
  • This could include theoretical linguistic descriptions of the feature and/or computational models of this feature as appropriate.
  • You should also name the specific language/s you wish to focus on, and why.
• The proposed modeling approach, with relevant citations - see references on the course page. The scope of the project should be clearly specified. Some questions to consider:
  • Will you train your own model, or use pre-trained resources? 
  • What, if any, aspects of model performance are relevant to your analysis? For example, should the model pass some accuracy threshold on a particular task, in order to support your evaluation?
  • Which aspects of model architecture do you anticipate being relevant to your analysis? For example, when working with a Transformer model, certain linguistic features may be encoded in earlier or later layers (cf. Tenney et al., 2019, linked on the course page).
• The initial hypotheses you plan to investigate, and what you expect to find.
  • An especially sound proposal would also include some contingency plans, of how you would handle cases where things go wrong - this often happens in research! For instance, if you plan to train your own model, you may also want to sketch out an alternative approach to your hypotheses using pre-trained resources, in case you run into problems training the model.
  • You may also want to outline follow-up directions for investigation, based on what you expect to see from your initial hypotheses. For example, "I expect that the model will have a shared representation for Feature A in both Language X and Language Y; however, if I find that is not the case, then I will [do something else; e.g. explore Feature B because I think it's related, or identify edge cases, or...]."

The proposal will count for 20% of your grade, so do spend some time on it, and email me if you have questions. Grade allocations and timelines have been updated on the course page: https://lacoco-lab.github.io/courses/ling-interp-langmod/