Responsible AI in research evaluation is not a slogan; it is a set of concrete policy choices a research office can document, defend, and audit. As AI features arrive in research-information platforms — author disambiguation, profile summaries, topic clustering, research-impact suggestion engines — the question of which decisions AI is allowed to drive matters more than which AI features the platform offers. This article gives research leaders a framework grounded in DORA, CoARA, and emerging institutional practice.
The principles to align with
Three policy reference points anchor responsible adoption:
- DORA (San Francisco Declaration on Research Assessment) — research is evaluated on its own merits, not via proxies. AI-derived scores are a particularly opaque proxy and should not drive individual evaluation.
- CoARA (Coalition for Advancing Research Assessment) — over 800 signatories committed to broader, context-aware evaluation. CoARA's working groups have begun publishing concrete tools, including the OI4RRA outputs formally endorsed in March 2026.
- Institutional transparency obligations — most universities now require documented model use in decision-affecting systems. AI in research evaluation is exactly such a system.
The decision framework
For any proposed AI feature inside a RIMS, ask four questions in this order. (1) What decision does this AI affect? If it is a researcher-affecting evaluation (hiring, promotion, funding allocation), the bar is highest. If it is a discoverability or workflow improvement, the bar is lower. (2) Can the output be inspected? A score that cannot be reproduced or explained cannot be defended in a hearing. (3) What is the human-in-the-loop? Manual override paths and researcher correction mechanisms are not nice-to-haves; they are the difference between AI as decision support and AI as decision maker. (4) Is the underlying data trustworthy? An AI feature on poorly reconciled data is worse than no feature at all — covered in AI is Only as Good as the Data Beneath It.
What good policy looks like
A defensible institutional policy for AI in research information typically covers: which AI features are in use, on which data, with what review process; which decisions AI may assist and which it may not drive; how AI outputs are labelled to researchers and external readers; how errors are corrected; and how the policy is reviewed annually as the technology evolves. The policy lives alongside existing research assessment policy, not replacing it.
Practical examples
Author disambiguation — appropriate AI use, because outputs are inspectable (you can see which records were matched), researchers can claim/reject, and the alternative (manual matching at scale) is impractical. Profile summaries — appropriate with labelling and researcher review. Topic clustering — appropriate for strategic planning, with awareness that clustering reflects training data biases. Researcher ranking — inappropriate. Hiring recommendation — inappropriate. Promotion-suggestion engines — inappropriate. The pattern: AI for description and discovery; reconciled metrics and human judgement for evaluation, in the spirit of our journal and researcher metrics pillar.
Building institutional capability
Responsible AI adoption is not a procurement exercise; it is a capability. Train research office staff to interpret AI outputs critically. Engage researchers in policy formation so they understand what is decided about them and how. Document everything. Treat the AI policy as a living document that you refine as the technology changes — and as CoARA and other community bodies publish further guidance.
Frequently asked questions
Is "responsible AI" just a slogan? It can be. Operationalised — documented decisions, transparent outputs, human-in-the-loop, audit trail — it is policy.
Do we need a separate AI policy? Either a standalone policy or an addendum to existing research assessment policy. The form matters less than the substance.
How does this align with our CoARA commitment? Directly. Responsible AI use in evaluation supports the broader-evidence, context-aware assessment CoARA signatories commit to.
Where to start
Discover RIMS provides the reconciled foundation — outputs, identifiers, collaboration, journal context — on which any responsible AI feature depends, and surfaces decisions transparently so that institutional policy can be defended in front of any panel that asks.