How We Conduct Research
Our research runs on Warren, our conversational survey platform, and follows the Warren Methodology Framework. The framework sets our standards for survey design, bias prevention, ethical and trauma-informed framing, validation, and data quality. A few commitments shape everything we do:
Layered, explicit consent. Taking part in one part of a survey never assumes consent to another. People consent separately to participate, to have their narrative responses used, to share demographics, to provide any audio or video, and to be contacted again in the future. Any layer can be declined, and any consent can be withdrawn at any time.
A named human reviews the analysis. Our platform detects patterns. A named analyst interprets them. Findings are reviewed by a person before they become conclusions, and where an engagement calls for it, that reviewer brings relevant community standpoint or subject-matter expertise.
Structural bias checks on every instrument. Every survey is reviewed against a structural bias typology that covers both individual response biases (such as social desirability and acquiescence) and structural features of the instrument itself (such as deficit framing and reference-category bias).
Trauma-informed and healing-centered design. For research that touches difficult experiences, participation is always optional, content warnings and supportive resources are provided around sensitive material, and questions are framed to honor agency and aspiration rather than only document harm.
How We Practice Transparency
Every Warren project produces a methodology disclosure aligned with the eleven standardized elements defined by the American Association for Public Opinion Research (AAPOR) Transparency Initiative. These disclosures document who sponsored and conducted the research, how participants were recruited, how data was collected and processed, how it was or was not weighted, and the limitations of each study.
We publish these disclosures within each project dashboard so that clients, participants, and reviewers can verify our methods. Where our methodology goes beyond what the AAPOR elements were built to capture, such as our validation work across the life of an engagement and the named-analyst interpretive layer, we make those commitments visible in the same place rather than keeping them out of view.
Methodology Framework
Current engagements follow the Warren Methodology Framework v4.0, published in plain language as Below the Noise. It establishes our standards for conversational survey design, bias prevention, ethical and trauma-informed framing, multi-phase validation, and data quality assurance. It is grounded in established survey research literature and behavioral science principles.
AI in Our Process
We are deliberate about where AI is used. The simplest way to put it: AI helps us design surveys and draft reports, but no AI ever interprets your responses.
Where AI helps (design and drafting)
Generating discovery questions and drafting technical project briefs during scoping. This works from the project's specifications, not from participant data.
Drafting survey instruments from approved specifications. These are then authored and reviewed by the project lead before deployment.
Drafting report narratives, executive summaries, and recommendation scaffolding. These are written only from structured, aggregated analytical outputs (such as theme counts, sentiment distributions, and archetype breakdowns), never from raw participant text. Any direct quotes that appear in a report are selected by our deterministic process, not by AI, and the project lead reviews and approves all content before it is delivered.
Where AI stops (your responses)
No AI, large language model, or machine learning system processes participant responses to produce analytical outputs.
Theme detection, sentiment scoring, quality scoring, and archetype classification are all deterministic and rule-based.
No respondent data is transmitted to any external AI service.
We never train AI or machine learning models on client or participant data.
A detailed AI usage audit documenting every instance of AI involvement in the Warren platform is available upon request.
Published Methodology Disclosures
The projects below were completed under earlier versions of our methodology framework. We preserve their disclosures as the accurate record of how that work was conducted at the time. Current engagements follow the v4.0 framework described above.
Greater Hartford Arts Council Donor Insights Survey
(conducted under Framework v2.0) Conversational donor intelligence survey conducted August to October 2025. 157 completed surveys from approximately 2,000 contacts. Web-based deployment via the Warren platform.
🔗 See Disclosure here
_______________________________________________________
Westover Student Experience Survey
(conducted under an earlier framework) Conversational student experience survey conducted December 2025 to January 2026. 28 completed surveys from approximately 200 invited students. Web-based deployment via the Warren platform.
🔗 See Disclosure here
How We Handle Your Data
Every dataset passes through Warren's data cleaning pipeline at the point of import. Every transformation is logged in a structured audit trail that records the original value, the new value, the reason, a confidence score, and whether a human reviewer could override it. We do not perform data imputation. We never fill in missing responses or infer answers on behalf of participants.
Row-Level Validation
Empty rows removed, duplicates detected, timestamps verified for impossible dates and out-of-range durations.Field-Level Cleaning
Whitespace normalized, encoding repaired. Narrative text is always preserved exactly as the participant entered it.
Spam and Low-Effort Detection
Blocklist filtering, repetition detection, and minimum effort thresholds flag responses for review.Response Normalization
Semantically equivalent responses grouped under canonical labels. Multi-select values parsed into structured arrays.
Quality Scoring
Each response scored 0 to 100 on length, effort, and coherence.
After cleaning, analysis is performed entirely by our deterministic, rule-based pipeline, which produces themes, sentiment, quality tiers, impact areas, level of action, and archetype patterns. A named analyst then reviews those outputs before they become findings.
Our Boundaries
✔️ We do not sell respondent data
✔️ We do not use respondent data to train AI models
✔️ We do not use AI to process, code, or analyze respondent data
✔️ We do not impute missing responses
✔️ We do not collect sensitive or demographic data without separate, explicit opt-in
✔️ We do not claim probability sampling when using non-probability methods
✔️ We do not report margins of error for non-probability samples
✔️ We do not validate our research against paid panels, bought survey takers, or focus groups
✔️ We do not contact minor or student participants directly. Engagement with minors is routed through their school or institution.
Questions About Our Methods?
We welcome scrutiny. If you have questions about how we conduct research, process data, or analyze results, we are happy to discuss. Contact us here.