Recipe 4

Turn your field's literature into data, and into a sparring partner¶

Try this first. Load your field's papers and your newest results into one workspace, and ask for a source behind every answer.

Summary¶

Agents can extract data from papers and old experiments into tables, with the source of each value. Examples are gene signatures, sequences and assay results. An expert can also load most of the literature of their field, add the lab's latest results, and question the agent as a sparring partner. Agents asked for ideas without that kind of steering produce many plausible hypotheses and few worth pursuing. Researchers stay in charge of their own questions and hypotheses.

Steps¶

Collect the papers, supplements and the lab's unpublished results. Use your institution's full-text access where you have it.
Ask for a table with one row per finding and a source and page for every value.
Check a random sample of rows against the papers by hand.
Check old findings for confounders that are known today.
Ask the agent questions about the collection, and require a source for each answer.

How to check the output¶

Every value has a source, so every value can be checked. A person checks a random sample of rows. A larger sample gives more confidence in the table. When the agent answers questions, the expert judges each answer.

Examples¶

Disease signatures were extracted into tables linked to the papers. Some older signatures turned out to reflect a confounder that was not known when they were published.
Small studies from past decades contain many data points that were never combined.
A group reanalysed old phenotyping data, and the results became a conference keynote.
A clinician-scientist loaded almost all papers in their field, with supplements, and recent results from their lab. Their questions to the agent produced new hypotheses and changed their research plans.

Known problems¶

Extracting data from figures in old papers is still unreliable. Check these values more often than values from tables or text.
Hypotheses generated by agents alone were the weakest use reported. Agents produce many plausible hypotheses quickly, and sorting them takes a lot of work. Published work finds the same: ideas from language models lose their apparent advantage once they are carried out. Hypotheses from an expert asking questions about a complete collection were among the best.
It is not yet clear which expert conversations produce new ideas and which mostly return what the expert already knew. Ideas generated by AI tend to stay close to the literature they start from.
A value without a source cannot be checked later.

Skills¶

Each skill page has a brief. Give it to your agent to build a version for your own institute.

literature-to-table: extracts values from papers into a table, with the source and page for every value.

Best practices on aiforscience.eu that this recipe follows:

Make provenance and citation first-class: every extracted value links to its source.
Write documentation and data for agents: without full-text access, the agent reads only abstracts.

Change history¶

2026-10-30: Renumbered to follow point 4 of the submitted Comment.
2026-10-28: Updated to match the presubmission draft of the Comment.
2026-09-25: Updated to match the first full draft of the Comment.
2026-09-25: First draft, from the first round-table.