How should law and regulation cope with fast changing technologies and industries? How should they balance the risks that come with new ideas and the risks of crushing them? And how should they help to ensure that the benefits of new technologies are widely spread?

In this blog (which follows on from one on the Fourth Industrial Revolution) I suggest that we are beginning to see a radical change in both the theory and practice of regulation with the emergence of a new field of ‘anticipatory regulation’.

From stable to iterative regulation

In recent decades, the dominant ideas about regulation emphasised that it should be constant, simple and predictable. If it was, markets could do what they do best, finding ways to optimise the implementation of new technologies and ideas.

A host of regulatory theory, and regulatory institutions, emerged from the 1970s onwards, to put the economic theory of regulation into practice. It adapted long-standing ideas about market failure, externalities and the risks of regulatory capture. It tended to emphasise the role of regulators in promoting competition and contestability more than a previous generation of ideas that were mainly about constraining the abuses of monopolists.

Its advocates promised to replace the capricious decisions of bureaucrats with more arm’s length rule-makers and more rational rules. The promised result would be more competition and a better deal for consumers. The idea was that regulators should not try to second guess the direction of technological change. Instead they should set the rules of the game, and then stand back.

Traditional regulatory theory still arguably works fairly well for stable industries with relatively stable technologies. But it struggles to cope with more fluid, dynamic and uncertain fields, particularly ones where the boundaries between industries are constantly changing. This is particularly the case with many of the new platform based business models that have come to dominate so many industries.

Regulators have always faced an inescapable dilemma on timing. Acting to regulate too early can kill off, or freeze, innovative business models with a potential for public good. Acting too late can leave consumers exposed to harm, or allow new monopolies to become entrenched.

In the past, regulators assumed that they could ignore new developments until they reached a certain scale. Likewise, new firms didn’t engage with regulators until they hit a large enough scale. But speed undermines both sets of assumptions. Small firms can become big very fast, much faster than in a predominantly material economy, and that’s forcing attention to quick and lean ways of linking what may be a large pool of potential new approaches and innovators to the limited resources of regulators.

The related challenge is that the structure of markets is radically changing with new forms of market power embodied in platforms. These tend to form quasi-monopolies; give great gains to consumers in terms of lower prices but also potential harms. Market dynamics can risk reducing competition and innovation as large incumbents are incentivised to buy out, and often shut down, potentially threatening incomers.

Such platforms also adopt a ‘frenemy’ approach to smaller providers, drastically squeezing their margins, and using algorithms to take advantage of asymmetries of information between sellers and buyers. Economics has struggled to make sense of these new dynamics, but it’s becoming less plausible that the benefits from dominant platforms outweigh the harms.

So, on the one hand, there’s a drive for more open and less restrictive regulation; and on the other, there are strong pressures for more intervention and scrutiny.

The rise of anticipatory regulation

Governments around the world are grappling with these questions. A family of new methods that can loosely be described as ‘anticipatory regulation’ is now emerging. These recast regulation to assist in the emergence of new technological tools while also allowing faster responses to ensure that the public aren’t exploited and that new dangers are averted.

Elements of this aren’t new. Smart governments have long tried to achieve a better alignment of technology development, market regulation and public policy, pre-empting major shifts, as Scandinavian governments successfully did with GSM mobile a generation ago. There are some good recent examples - in fields such as human fertilisation - of imaginative exercises to address the intertwined issues of ethics, public confidence and technology development in emerging fields, deliberately anticipating potential problems.

But in other respects the tools are new and are evolving to navigate economies in which:

  • the pace of change is rapid, as Moore’s law continues to operate and is matched in other fields like genomics;
  • barriers to entry are often very low;
  • ability to operate across sectoral boundaries is high (with technologies like 3D printing, platforms);
  • business models encourage big firms to operate across multiple sectors (as Google/Alphabet now does) or run whole ecosystems of products and services (like Apple or Amazon);
  • market dynamics depend much more on data and algorithms than the traditional tools for asserting power;
  • new capabilities in machine intelligence are throwing up a host of legal, ethical and practical challenges, which traditional regulatory and legal concepts simply don’t fit.

Ten elements together make up the emerging toolkit for governments and regulators. Here I briefly describe each one.

1. Open dialogue with innovators as well as incumbents

The first shift is towards more explicit and open dialogue with innovators and entrepreneurs to ensure that regulations are not blocking them in ways that offer little public benefit. The finance sector has been a pioneer in this respect – with a more open dialogue between regulators and newcomers in the UK, Singapore or the US as examples. Similarly there is growing interest in open dialogue with potential entrants involved in BitCoin and Blockchain technologies by regulators aware of the comparative advantage that could be offered to London, New York or Dubai from being ahead of the curve.

The United Arab Emirates (UAE) has attempted to spread this idea to other sectors through what it calls ‘government accelerators’ – for example, bringing in start-ups to work on reducing traffic congestion, road accidents or air pollution in close collaboration with government officials.

Other regulators have tried to act in advance of shifts in technology after dialogue with industries. In the US, for example, the National Highway Traffic Safety Administration developed policies on autonomous vehicles in 2013 to pre-empt their widespread introduction and worked with industry to better understand how driverless cars and driven cars would interact.

2. Iterative not definitive rules

Next, these methods imply that regulation should often be iterative rather than definitive. The benefits from continuous adaptation may outweigh the benefits of stability and predictability (though of course there will be trade-offs).

As a result, regulators are trying to work as much as possible through guidelines; promoting self-regulation and understanding, through communicating directions of travel more than specifics (for example, of how wearables might be regulated in the future). The rapid pace of development of new mobile application initiatives is a case in point, which prompted the Food and Drug Administration in the US to create a set of guidelines to regulate the health and wellness applications.

3. Testbeds and sandboxes

Regulators then try to help innovators try out their ideas, and think through the regulatory implications. Finance has, again, arguably led the way. Organisations like the UK’s Financial Conduct Authority use regulatory sandboxes to allow new entrants to test out their products, and the potential regulatory implications, in a close dialogue with policymakers. They also commit to changing or adapting regulation in response to new entrants.

This has been one of the factors that has allowed the alternative finance sector to grow – with crowdfunding, peer to peer and other tools, each of which posed challenges for regulators.

Sandboxes vary from the purely advisory, through a focus on current products and services, to genuine anticipation of potential services. Several dozen are trying to set up FCA-style sandboxes, helped, for example, by R2A, the RegTech for Regulators Accelerator which is focused on innovations in financial regulation.

A parallel approach is emerging in other sectors. For example, Catalonia provides testbed facilities for autonomous vehicles, linking car manufacturers (Seat, Nissan), industry representatives (Ficosa), telecommunication companies, academia and legislators (e.g. transportation service and the Mayor of Barcelona).

The Innovation Testbeds in Korea are another good example, using residential areas to speed up useful innovation in the Internet of Things (IoT), again with the aim of enabling close communication between innovative firms and policymakers.

Image: Nesta

4. Risk management rather than risk avoidance

The idea behind sandboxes and test beds is risk management – to test out new ideas in safe environments that minimise negative risks but also make the most of positive risks.

A related concept is risk-based regulation and inspection - using data, and predictive tools to better map where problems are likely to arise so as to economise on scarce regulatory resources. Establishing parameters within which emerging technologies can operate safely helps to release new products to the market and creates a sense of predictability and certainty for the industries. It also makes new industries more attractive to potential investors.

The European Commission and the US Food and Drug Administration, for example, now include permission clauses for new tech that doesn’t fit existing frameworks to be released to the market in controlled ways, with data requirements so that results can be tracked.

5. Regulators driving innovation directly

A potentially significant shift is direct involvement of regulators in innovation – using orchestrated innovation to achieve regulatory goals such as greater competition.

Nesta is directly involved in one example of this through the Open UpChallenge, which has been developed with the Competition and Markets Authority and the major banks, and aims to link the opening up of SME’s data held by banks, a drive to improve competition, and concerted moves to finance accelerated innovation, to ensure that new products and services are available to make full use of open data.

In this case the regulator required the incumbents to fund open processes for innovation that could threaten their market share, amplifying competition and accelerating the development of new technologies and business models. This kind of approach may spread to other similar industries. Energy and housing are obvious examples.

6. Joined up regulation

Many of the challenges come from changes that cut across sectoral boundaries. The 1980s regulators were very sectoral – electricity, post, phones, gas – and assumed clear boundaries.

The convergence of communications technologies has been threatening this for decades – and justified the creation of OfCom, for example. That convergence has spread; for example, Singapore merged the Info-Comms and Media regulators in 2016. Telecommunication companies now provide banking transactions, such as Safaricom and M-PESA in Kenya, tech giants are getting involved with the grocery industry, with Amazon buying out Whole Foods, and Apple is directly involved in money.

These trends are potentially going much further with the Internet of Things, and the further integration of the data economy and material networks in transport, energy and buildings. All are coming up against very similar challenges around data, monitoring and maintenance of infrastructures and protecting against cyberattack.

7. Public engagement

A practical challenge for regulators is how to interact with stakeholders at scale, rather than just talking to a handful of big firms. Here there are interesting experiments that try to involve many more people, who can be engaged in shaping regulations around emerging industries – like vTaiwan run by the Taiwanese parliament (and described in the recent Nesta study on digital democracy). France’s open process for drafting its digital law is another example, and pulled in 20,000 contributors.

The growing field of digital democracy is becoming much more sophisticated in understanding how deliberations can be inclusive, structured and staged to ensure a better grasp of the issues, to prevent capture by powerful interests, and to ensure both better diagnosis and prescription.

Given the huge implications, and ethical issues, surrounding next generation AI, genomics and transport technologies, one of the biggest risks for industries is to attempt to bypass public engagement and legitimation. History shows that this invariably backfires.

8. Explicit politics - it’s not just technical

A related point is that anticipatory regulations cannot avoid politics. There is no pure rational calculus for deciding how a new industry should develop, or how a new technology should be used (attempts to assert a purely technocratic approach have tended to flounder). Instead many factors intersect - economics, ethics and politics (as I set out in more detail in my recent paper on good and bad innovation). Problems are bound to arise if any of these are ignored and regulation is seen only through a technocratic lens.

Open data and transparency can help to politicise regulation in healthy ways. In the US, the Community Reinvestment Act did this for banking several decades ago – bringing to the surface the many distortions and biases in lending behaviour. Deloitte’s Regulators of Tomorrow report gives the example of how Zanesville residents of predominantly African-American descent won over $10m settlement after using open data to prove that water supply companies were discrediting against them in favour of the ‘white’ neighbourhoods.

The key point is that there will often need to be some political or ministerial engagement in the big choices that arise out of the various methods described above – addressing winners and losers, potential risks and harms as well as gains.

9. Proportionate complexity

For decades, it has been assumed that good regulation should be as simple as possible. In the 1990s and 2000s, various governments (The Green Deal in the Netherlands, for example) set quantitative targets for cutting the volume of regulation, which was seen only as a burden on SMEs. The EU Commissionreported that it achieved a 25 per cent decrease of administrative burden.

The UK Better Regulation Executive adopted a ‘one in two out’ rule for similar reasons. But the anticipatory regulation logic may point in the opposite direction, towards more complexity - ideally with simple principles but flexibility to devise sufficiently detailed regulations to enable new models to emerge.

10. New skills for regulators

To fulfil these new tasks, regulators need new skills – more in-depth understanding of new technologies and business models - perhaps more than the theoretical economic and legal knowledge which was prioritised in a previous era.

Another shift is to go further in moving from a primary focus on process compliance towards a greater focus on outcomes. More than ever, too, regulators need communication skills – but now not just the skills of handling coverage in specialist media, but also the ability to engage with large communities of stakeholders.

Over the next year we’ll be looking at these emergent models of anticipatory regulation. How are these organised? What skills do they need? How do they strike the right balance between breadth and depth? How will they differ their function in established vs disruptive industries? How are they funded? How do they ensure the right input from stakeholders? What can other sectors learn from the methods being pioneered in finance? In particular, how could more experimental methods be used to find out what works best?

Drones and machine intelligence test-cases

Drones are a particularly good example on which we have been working for some time with cities, businesses and governments. They offer huge potential gains in terms of speed and cost, but also big challenges - how to handle routes, data, pricing models, environment effects and so on.

Many of these challenges can only be worked out through real-life testbeds in which providers and policymakers work together closely, and ensure that any solutions are acceptable to the public. Nesta is currently working with a range of cities, governments and companies on what this means in practice.

Another very live example is the rapid spread of algorithms throughout the economy and daily life. I’ve written previously about the case for a Machine Intelligence Commission – and the debate is slowly limbering up on possible new institutions that can help to navigate a wise course, making the most of new tools while avoiding the worst pitfalls.

Nesta is also working with partners across Europe on new data commons, which could also become an important part of the landscape for transport and other infrastructures, and part of the regulatory picture.

We may need quite new kinds of institution to handle options which are increasingly being debated, such as mandatory data-sharing, or prohibitions on uses of certain classes of data, or, as in the work of Glen Weyl and others, rewards for the public who generate data in the first place.

Next steps

The field of anticipatory regulation is beginning to take shape. There are few professionals in anticipatory regulation or handbooks; and not much theory. There are some promising studies, such as this recent one from Deloitte, and growing interest in regulatory innovation. And there are some very interesting publications emerging on the boundaries of economics and law, such as Ariel Ezrachi and Maurice Stucke’s recent book on virtual competition.

But this is a field waiting to be shaped, which makes it all the more interesting and exciting.