Founding Scientist-AI Review of Biomedical Grants & Papers for CoPAIR (Collaborative Peer AI Review)

Founding Scientist — AI Review of Biomedical Grants and Papers

CoPAIR (Collaborative Peer AI Review) · Full-time · New York City (hybrid, in-office majority). Relocation supported

About us

CoPAIR LLC is building an AI system that evaluates scientific manuscripts and grant proposals with the rigor of an expert review panel. Rather than treating a single model as a “reviewer,” we orchestrate ensembles of frontier models in specialized, complementary roles and calibrate them as a measurement instrument — one whose judgments are validated against real funding and publication outcomes, not assumed.

We work with leading research funders and journals, are advised by a scientific board drawn from Stanford, Harvard, and UCLA, and are affiliated with Stanford HAI. We are founder-funded and moving quickly. The mission is consequential: improving how science gets evaluated, funded, and published.

The role

This is a ground-floor scientific hire reporting directly to the founder. You will help design, build, and — critically — validate the system that decides how good a piece of science is. Day to day, you will:

• Build the evaluation pipeline. Develop and extend our multi-model, persona-based architecture that reviews proposals and manuscripts for impact, rigor, and risk.
• Own the validation science. Design and run the criterion-validity, construct-validity, and reliability work that measures our system against curated corpora of real grant and publication outcomes. Where do we agree with expert panels, where do we diverge, and why?
• Run experiments and ship improvements. Turn disagreement analysis, calibration studies, and failure cases into concrete changes to prompts, personas, and scoring.
• Translate domain expertise. Work directly with our scientific advisors and funder/journal partners to encode what good science actually looks like into evaluation criteria.
• Set the standards we hold ourselves to. Help define how we judge our own instrument — the benchmarks, the contamination controls, the metrics that matter.

Who you are

• PhD or equivalent experience
  in a quantitative life science, computational, or related field — or a strong metascience / research-evaluation background.
• Fluent in Python, comfortable building, debugging, and reasoning about data and ML pipelines.
• Hands-on with modern AI tooling — LLM prompting, orchestration, and (especially) rigorous evaluation.
• Strong on experimental design and statistics. You think naturally in terms of validity, reliability, confounds, and measurement.
• A real reader of primary literature. You can pick up a biomedical paper outside your subfield and judge whether the methods support the claims.
• Comfortable building from zero in an ambiguous, fast-moving environment.

Nice to have

• Firsthand experience of peer or grant review — as author, reviewer, or panelist.
• Background in metascience, psychometrics, or research-on-research.
• Experience benchmarking or red-teaming LLM systems.
• Prior startup or zero-to-one experience.

What we offer

• A genuinely novel problem at the intersection of AI and metascience — work that could change how research gets funded and published.
• Direct collaboration with the founder and a world-class scientific advisory board.
• Real ownership and influence over the scientific direction of the company.
• Competitive salary in the range of $170,000–$215,000, depending on experience, plus meaningful founding-level equity.
• Based in our New York City office, with a hybrid schedule weighted toward in-person — especially over the first year, when alignment on how we evaluate science matters most. Relocation support available.

How to apply

Send your CV and a short note on why this problem interests you to avillapiano@copairllc.com. Optional but welcome: point us to a paper you’ve reviewed — formally or informally — and tell us what you think the field got right or wrong about it.

CoPAIR LLC is an equal opportunity employer.