Lesson 77 · The Grant Architect

77. AI Spotlight

30 min

By the end you'll be able to

  • Use AI to accelerate drafting of evaluation matrices, survey items, and dissemination plans.
  • Identify the elements of an evaluation plan that still require human validation.
  • Recognize common AI failure modes in evaluation work.
  • Explain to a program officer how the team uses AI responsibly.

Artificial intelligence is already changing how evaluation plans get drafted, and the teams that learn to use it well are saving days of work per proposal. In this lesson you learn how to use AI as a drafting partner for survey items, outcome indicators, evaluation matrices, and dissemination plans, while keeping a clear-eyed view of where validation is non-negotiable.

You will work through a structured prompt for drafting an evaluation matrix from a logic model, a prompt for generating candidate survey questions tied to a validated construct, and a prompt for stress-testing your evaluation plan against common reviewer objections. You will also see the failure modes: AI confidently inventing instrument names, citing studies that do not exist, and producing survey items that are double-barreled or leading. Each of these is a credibility risk if it reaches a reviewer unedited.

By the end you should be able to use AI to accelerate the drafting stage of your evaluation plan, name the specific elements that still require human review (instrument selection, IRB language, statistical methods, citations), and explain to a program officer how your team uses AI responsibly. Used well, AI compresses the timeline. Used carelessly, it produces evaluation plans that fail on the first careful read.

Common mistakes

These are the traps learners hit most often on this topic. Knowing them in advance is half the fix.

  • Submitting AI-drafted evaluation plans without methodological review.

    AI drafts read fluently, which makes the methodological gaps invisible to a non-expert reader but glaring to a reviewer.

  • Accepting AI citations at face value.

    AI routinely fabricates plausible-looking citations, and a single invented reference in an evaluation plan can undermine the credibility of the entire proposal.

Practice problems

Try each on paper first. Click Show solution only after you've made a real attempt.

  1. Problem 1
    Draft a prompt that asks AI to generate candidate survey items for measuring program-driven changes in self-efficacy.
    Show solution

    Generate eight candidate Likert-style survey items measuring self-efficacy in adult participants of a workforce training program. For each item, note whether the wording overlaps with the General Self-Efficacy Scale, the New General Self-Efficacy Scale, or other validated instruments, and explain why the item is or is not double-barreled. Do not invent instrument names; if you are not certain a scale exists, say so.

Practice quiz

  1. Question 1
    Which task is AI best suited to in evaluation work?
  2. Question 2
    What is a common AI failure mode the lesson warns about?
  3. Reflection 3
    In two sentences, explain why instrument selection still requires human review even when AI drafts the survey.

Lesson 77 recap

AI compresses the drafting timeline for evaluation plans, but instrument selection, IRB language, statistical methods, and citations still require human validation.

Coming next: Lesson 78 — Introduction to Federal Cost Principles

That closes Week 7 on evaluation. Next week we move into budgeting fundamentals, where the evaluation budget you drafted here becomes one line item inside a larger financial story.

Saved in your browser only — no account, no server.