8. Designing for Complexity

Matching design to problem

Research design is supposed to be a function of the question. In practice, design often follows training: a researcher trained in RCTs runs RCTs, a researcher trained in ethnography runs ethnographies. This produces work that fits the researcher rather than the problem.

The transdisciplinary discipline is matching design to the problem's complexity. Some problems are complicated (many parts, knowable answer) and call for linear designs. Some are complex (interacting actors, emergent behavior) and call for adaptive, multi-method, or systems-aware designs. Mixing them up is a methods error.

Linear vs. adaptive design

A linear design locks the protocol before data collection and runs it through. Strengths: reproducibility, clarity, sample-size planning. Weaknesses: when the problem turns out to be different than expected, the protocol can't accommodate.

An adaptive design specifies in advance what would trigger a design change and what the change would be. Pre-specified flexibility is the key phrase. Strengths: responsiveness to emerging information without losing rigor. Weaknesses: takes more design work upfront, requires governance for change decisions.

Adaptive designs are not "we'll figure it out as we go." That's drift, not adaptation. The rigor of an adaptive design is in the pre-specification:

• What triggers a change? (Defined criteria — recruitment lag, emerging theme, drift in baseline characteristics.)
• What change is licensed? (Specified, not open-ended.)
• Who decides? (Pre-named decision-makers and process.)
• How is the change documented? (Audit trail.)

Without these four, "adaptive" is just opportunism.

Validity across paradigms

Mixed-method studies need validity concepts from each paradigm in play. Importing one paradigm's validity concepts wholesale produces unfair appraisal of the other strand.

• Internal validity (positivist) — does the design support a causal claim?
• Statistical conclusion validity (positivist) — is the statistical test appropriate, powered, and correctly interpreted?
• Construct validity (post-positivist) — do the measures capture the constructs they claim to?
• External validity / generalizability (post-positivist) — to what populations and contexts do findings extend?
• Credibility (interpretivist) — do findings ring true to the phenomenon and participants?
• Transferability (interpretivist) — can readers judge whether findings apply to their context, based on thick description?
• Dependability (interpretivist) — is the analytic process documented for audit?
• Confirmability (interpretivist) — are findings grounded in data rather than researcher preconceptions?
• Catalytic validity (critical/participatory) — does the study advance change for the community studied?

A mixed-method study addresses the validity concepts of each strand. You don't average them; you honor each.

When complexity requires methodological integration

Some problems can be addressed with a single method coherently. Others can't — the integration of multiple methods is constitutive of the question.

Signs that integration is required:

• The question explicitly asks about both magnitude and meaning ("how widespread is X, and what does it mean to those experiencing it?")
• Mechanism is unclear — quant findings need qual to explain heterogeneity
• The phenomenon is contested — different stakeholders define it differently, and you need each definition's evidence
• Generalizability claims depend on understanding context
• Action is the goal, and the action requires both magnitude (for scale) and meaning (for buy-in)

When these signs are absent, a single-method study may be the cleaner choice. Mixed isn't always better — it's better when the question demands it.

Mixed in name only

A common failure mode: a study that calls itself mixed-methods because it includes both a survey and interviews, but reports them in separate sections that never inform each other.

Test: can you delete one strand and still have the same conclusions? If yes, the strands weren't integrated. The point of mixed-methods is that the integrated conclusion is unavailable from either strand alone.

The fix is at the design stage — specifying integration points (sampling, analysis, interpretation) and integration products (joint displays, meta-inferences) before data collection. We'll cover these in Module 3.

Adaptive design in practice

A practical example. A team is studying a community health worker intervention. The pre-specified adaptive triggers:

1. Recruitment lag: if enrollment is below 80% of plan at 8 weeks, the team will expand to a second clinic site (specified in protocol).
2. Emergent theme: if interviewers report a recurrent participant concern not in the guide, the guide will be updated and prior transcripts re-coded with the new probe.
3. Differential dropout: if dropout exceeds 20% in any demographic stratum, the team will pause new enrollment and convene the community advisory board to diagnose.

Each trigger is documented, each change is logged, and the final report describes which triggers fired. The study is more rigorous for having the triggers, not less.

Coherence in multi-method studies

A multi-method study is coherent when its strands inform each other and reach a defensible integrated interpretation. Coherence has three layers:

• Design coherence — the methods are chosen because they answer different facets of the same question
• Sampling coherence — strands sample purposefully relative to each other (e.g., qual sample drawn from extreme-scoring quartiles of quant strand)
• Analytic coherence — analysis includes integration points where strands speak to each other

Without coherence, multi-method is just parallel mono-method studies bundled together. With it, the integration is the contribution.

Closing

Match design to problem complexity. Linear works for complicated; adaptive is for complex problems where pre-specified flexibility is rigorous. Validity concepts come from each paradigm in play and aren't interchangeable. Mixed-method studies must integrate, not just co-occur. Coherence shows up in design, sampling, and analysis.

Next: experimental and interventional research in a transdisciplinary frame — applying design thinking to study design.