Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networks

Research Questions

Do LLM agents align more closely with human behavior when given only demographic information, or when supplemented with human belief networks?
Can providing a single belief “seed” improve human–LLM alignment across related topics?
How does the structure of belief networks influence the accuracy with which LLMs imitate human viewpoints?

Using demographic information alone did not produce meaningful human–LLM alignment.
When agents were given a single belief seed, alignment improved substantially for topics connected within the belief network.
No improvements were observed for topics outside the seeded network.
The degree of alignment increased proportionally with the factor loadings within the belief network.

Demographics Are Insufficient:
- Role-playing based solely on demographic cues failed to align LLM outputs with human beliefs in a meaningful way.
Impact of a Belief Seed:
- Providing one belief input improved alignment across conceptually related areas, indicating that LLMs propagate structured belief patterns when appropriately seeded.
Limited Alignment:
- For some issues—e.g., the death penalty—alignment remained zero even when the model produced the “correct” stance, highlighting disconnects between correctness and human-like belief patterns.
Structural Dependence:
- Alignment varied with the strength of connections in the underlying belief network, showing that LLM imitation depends on how beliefs co-vary among humans.
Ethical Risks:
- Because harmful or false beliefs can also be simulated, the approach carries risks of manipulation, misrepresentation, and ideological reinforcement.
LLM Models: 5
Synthetic Data: 3
Method: 4
Speed: 1
Ethics: 4
Accuracy: 4
Demographics: 3

If you would like to explore this research in more detail, click here to read the full paper.