mass peer review

A mass peer review process amplifies the power of peer review as used in the sciences by including many more people in the process. It has proven surprisingly robust in the open source software and open content movements. The GFDL corpus including all of Wikipedia attracts readers and editors from all over the world, thousands participating in any given year.

of public decisions

Any public decision involves similarly widespread consultation, e.g. using a model such as a Citizens Assembly or the good old political party to refine some positions for presentation to the public. However to reconcile democracy with expertise has historically been very difficult and has to some degree led to the schism we today call right versus left, which gain authority from experts or mass approval respectively, and to today's representative democracy.

Since politics itself as we understand it today has not solved major problems, e.g. climate, peace, poverty, etc., and there are more serious problems coming, e.g. pandemics, dangerous technology, many people think this old model is not good enough. See also green politics.

problems of scale

Extending decision making models that work on a small scale, notably consensus decision making and democratic structuring, to accomodate masses of people in a participatory democracy has proven problematic.

Ordinary peer review relies on a very small number of highly qualified, credentialed, and often dedicated reviewers. Expanding this into an effective and ethical deliberative democracy of many people making arbitrarily complex decisions and carrying them out brings on many challenges:

logistical challenges

  • efficiently presenting the material and rapidly incorporating feedback, for which wiki technology is ideal
  • making effective use of anonymous trolls and the fly-by user
  • collaborative editing as a process - what list of process terms to employ in talking about the process of editing itself
  • organizing discourse about the material in such a way as not to obscure the material - for which talk pages are the standard
  • massively parallel collaboration so people can participate on multiple aspects of a project simultaneously, e.g. have many people working on writing all pages of a book simultaneously - giving rise to edit conflicts and point of view problems often best resolved with a neutral point of view standard.

For example, Mozilla uses a wiki to create documentation for Firefox and Thunderbird - any user can contribute. Among such a well-defined group of users, all of whom use the same technical artifact and all of whom can verify system behaviour in the documentation easily on their own, and in which naming conventions are well defined by name precedents in the user interface of the software itself, most users can do this without too many clashes.

editorial challenges:

Working as journalists or pundits or reporters of real world situations in a more general open content effort makes mass peer review more important and difficult:

Any and all of these processes require very frequent peer involvement and correction - it is estimated that simple maintenance of pages occupies over 30-50% of the time of most Wikipedia editors. The estimate for openpolitics.ca itself may well be similar at least for senior editors.

ethical challenges

A chief editor faces significant issues simply in defining what "peer" means - see senior editor guidelines.

These presents even more challenges in an explicitly or implicitly political wiki which focuses on a very large scale type of decision making and collaboration with very high regrets and cost of regrets, often in the thousands of lives or billions of dollars, e.g. e-government.

With such large scale decisions one must expect politics as usual to prevail and for factions to form any time such issues are discussed. The model of open politics itself emerged largely to deal with real world faction problems in a Canadian federal political party - see Living Platform in Practice.

political challenges

When peer-reviewing extremely contentious artifacts such as a party platform the ethical problems become overwhelming due to timelines, coordination and varying mission and goal statements. Sorting these out is a challenge that no one has yet solved, though there have been serious attempts to do so - see Green Party of Canada Living Platform and Imagine Halifax which resulted in drafts of planks and a citizen questionnaire respectively.

Mass peer review of this type may require:

Uses of mass peer review on this scale is studied under the names:

The set of ideas called open politics itself has become associated with best practices of all the above.

open politics in force

A more comprehensive auditable ruleset called open politics in force has emerged from practice in all of the above. openpolitics.ca itself has not yet adopted this ruleset but is considering it - defining OP-recognized credentials is a start.


A pre-requisite to implementing such rules is a reflexive intranet which is governed from within - that is, where a participatory democracy model prevails, and mass peer review extends to the web service's own rules of engagement between users and the social networks they form, typically called factions.


While rules merely provide guidance, some expect them to create the conditions from which a genuinely open politics can emerge: where mass peer review extends beyond written documents to actual decision and plan and goal statements, and where the situated effective action implied by a political decision is fully specified by reflexive commit verbs within the ultra-reflexive intranet formed by the many mass-peer-reviewed statements made in common by factions in the power network.


Some propose that a mindset must emerge in turn from that process where decisions are tied inexorably to lists of bodies they affects as victims and bodies committed to carrying it out: The Embodiment.

Given the extreme challenges to power structures involved, this may require a somewhat revolutionary interim, the Troll Age, wherein etiquette evolves fast enough that there are disturbing shifts in expectations and perhaps backlash from powers that be. As there were in both The Renaissance and The Enlightenment.