The Quality Project

The Quality Project at the City University of New York’s Craig Newmark School of Journalism, in association with the World Economic Forum, will evaluate and aggregate signals of quality in news, bringing together data from many disparate initiatives to make this information more useful for platforms and advertising networks and agencies, informing their decisions about ranking and advertising buys so they may give more promotion and financial support to quality, while withdrawing support from disinformation online. 

The Quality Project will provide a comprehensive set of quality gauging data to its chosen users so they select and weigh it in their own decision making process and algorithms. The project has identified the following categories of quality signals to be aggregated:

1. Observed. Trust Metrics’ staff and algorithms will immediately rate thousands of news sites against various criteria.

2. Self-reported. The Trust Project, Reporters Without Borders, and others are recruiting news organizations to sign onto various standards, working with W3C on data formats. This will require some level of auditing of compliance.

3. Behavioral. Various third parties — e.g., fact-checkers — monitor the work of news organizations and their output will be collected to inform ratings of sites.

4. Diversity. The Quality Project will note the communities and markets that various news sites serve to ensure a diverse list of sources.

5. Public opinion. The public’s view of and trust in various news sites is a useful signal if not used in isolation. The Reuters Institute for the Study of Journalism is involved with discussion to expand its research in surveying public opinion on news brands.

6. Endorsement. If a site is a member of an organization that enforces entrance standards or if it has won prizes from recognized authorities, these are additional signals.

7. Alerts. If a site distributes disinformation, conspiracy theories, and the like as identified by organizations such as Data & Society and the EU disinformation initiatives, this is valuable information to be collected.

Through the World Economic Forum, the project will convene stakeholders from journalism, media, technology, advertising, government, NGOs, and others in ongoing work to support the formulation of the most comprehensive source of quality signals for content available in the ecosystem.