Lesson 163 · The Grant Architect

163. The AI "Red Team"

30 min

By the end you'll be able to

  • Build a three-layer red team protocol: rubric load, persona panel, and synthesis pass.
  • Score a proposal against the funder's actual scoring criteria, not generic heuristics.
  • Identify the weaknesses that appear across multiple reviewer personas.
  • Decide which AI critiques to act on and which to discard.

The AI red team is the highest-leverage review move available to a grant professional working alone or on a small team. You cannot always assemble three human peer reviewers a week before submission. You can always run a simulated review panel that scores your proposal against the actual rubric, surfaces the weaknesses you stopped seeing, and produces a critique synthesis you can act on.

You will build a red team protocol in three layers. First, the rubric load: paste the funder's scoring criteria verbatim and instruct the model to score against those exact criteria, not against generic "good proposal" heuristics. Second, the persona panel: run the same proposal through three to five reviewer personas tuned to the funder context (a program officer, a methodologist, a community reviewer, a finance reviewer, a skeptic), and collect each critique separately. Third, the synthesis pass: instruct the model to identify the three weaknesses that appear across multiple reviewer personas, because those are the weaknesses most likely to cost you points in the real review.

The discipline is what you do with the output. A red team critique is a hypothesis about how reviewers will respond, not a prediction. You weigh each critique against your own judgment and your understanding of the funder, you address the weaknesses that are real, and you ignore the ones that misread the program. Used this way, the red team turns AI into the most patient, least expensive reviewer on your team.

Common mistakes

These are the traps learners hit most often on this topic. Knowing them in advance is half the fix.

  • Generic critique requests.

    "Critique this proposal" produces generic feedback. The rubric must be in the prompt.

  • Acting on every critique.

    Some red team critiques will misread the program. The human filter at the end is what makes the protocol useful rather than chaotic.

Practice problems

Try each on paper first. Click Show solution only after you've made a real attempt.

  1. Problem 1
    Write the rubric load prompt for an AI red team review of an NSF proposal.
    Show solution

    Act as an NSF panel reviewer. Score this proposal against NSF's two merit review criteria, Intellectual Merit and Broader Impacts, exactly as stated in the most recent Proposal and Award Policies and Procedures Guide. For each criterion, provide a numeric score on a five-point scale, a one-paragraph critique, and the two most significant weaknesses. Do not add criteria that NSF does not use.

Practice quiz

  1. Question 1
    What is the purpose of the rubric load step in an AI red team?
  2. Question 2
    Which weaknesses deserve the most attention in the synthesis pass?
  3. Reflection 3
    In one or two sentences, explain why a red team critique is a hypothesis rather than a prediction.

Lesson 163 recap

A three-layer red team (rubric load, persona panel, synthesis) is the highest-leverage review available to a small team. The output is hypothesis, not prediction.

Coming next: Lesson 164 — Data Privacy and Policy

Next, we close the week with the data privacy and policy discipline that protects everything the workflow touches.

Saved in your browser only — no account, no server.