As Europe marks the 30th anniversary of the fall of the Berlin Wall, the Middle East marks 40 years since the Islamic Revolution in Iran. In both regions, the changes in states and coalitions since these events have been profound.

In Europe, while many have retained their borders and constitutions, they have accommodated huge numbers of migrants with lasting impact on their domestic politics. Other states have ceased to exist, and some come into being, while others have revolutionised their economic and political models.

In the Middle East and North Africa, the changes have not been to borders or, largely, constitutions, but to security (in too many cases detrimentally) and society. The seemingly monolithic Ba’ath Party of Iraq has vanished, that of Syria been enmired and irretrievably diminished by a brutal civil war, and Muammar Gadhafi’s eccentric dictatorship in Libya has disappeared.

In recent years, the pace and nature of change in the Middle East have altered. While many of the challenges that existed 40 years ago persist (a young and burgeoning population, foreign interference, political divisions and above all the militarisation of society), new programmes and new voices are being heard.

Although there may still be calls from outside the region for ‘regime change’ in Iran and Syria, the voices inside the region are calling with varying degrees of passion for ‘state change’. It is less about who governs the country, but rather how it is governed. That is apparent in the reform agendas of stable countries, the current wave of popular protests and the challenges that war-torn countries face in rebuilding the state.

Equitable political structures

In the Gulf, ambitious programmes of economic and social reform that will change the structure of Gulf economies and the economic lives of their individual citizens are gathering pace. The flotation of a small percentage of Saudi Aramco is an iconic as well as financial moment, representing a new approach both to the Kingdom’s management and distribution of its wealth. So too is the liberalisation of the role of women in the Kingdom and the opening of public spaces to performance and artistic expression.

Similar ambitious plans in the UAE for diversification and development – including, in particular, indigenisation of the workforce – represent a shift from an economic model that has served the region for the past 50 years to one intended to enable it to remain globally competitive over the next 50, but which will redesign the social and economic roles of individual nationals.

Biggest challenge

Challenging as this will be, the greater challenges in the Gulf are the constitutional and social changes that an open and diverse economy will necessitate: not just for the maintenance of external confidence but also for the accommodation of a generation of citizens used to digitally enabled free expression and association. Some of the reforms themselves, most notably taxation, are historically associated with the reciprocal demand for representation.

The creation of equitable political structures that support and enhance economic reforms will become more pressing as the reforms take hold and change the economic life of citizens.

In Iraq and Lebanon, public mobilisation has been almost unprecedented in protest against the failure of governments to deliver and, crucially, serve. Accusations of incompetence, corruption, foreign interference and self-interest have all been hurled against incumbent governments. Representation, accountability and transparency are the basic and common demands from populations frustrated by the cumulative failures of the various forms of government that have evolved over the past decades. The calls are given additional passion by the preponderance of young people demonstrating and their rhetoric of a lost or stolen future.

It is axiomatic that where states fail to provide, others will organise.

An additional dimension to the crises in Lebanon, Iraq, Syria and Libya is that none of the governments control their sovereign territory or are independent of foreign intervention. It is axiomatic that where states fail to provide, others will organise. The proliferation of militias with a social role is testament to local resourcefulness, but also desperation in the absence of a competent local or national authority. It also represents an opportunity for criminality and violence, as well as opportunistic patrons such as Iran. Its sprawling network of influence across the region has had least purchase in states that have been ready to adapt and adjust their policies to include their Shia populations and exclude Iranian projection.

Innovation in the face of instability

Across the Arab world, the forms of government that have dominated in the last 40 years risk obsolescence. In three Arab countries – Syria, Yemen and Libya – states must be rebuilt after gruelling civil wars. Their challenge is to provide a form of government that is sufficiently inclusive and effective to allow them to move forward and avoid a recurrence of the grievances and conditions that led to war.

The chances of this in Syria with Bashar al-Assad in place and Russia and Iran acting as brokers are not good, and in Yemen the influence of foreign powers, in particular Iran, will have to be disentangled from the mosaic of domestic players. In Libya, none of the current options are sufficiently innovative. A settlement that does not break with precedent and does not address contemporary expectations will not hold.

In Egypt, the balance between a strong central government able to deliver security and economic growth in the short term and, on the other hand, a more representative and inclusive model to make both stability and growth sustainable will have to be adjusted in favour of the latter. Algeria, meanwhile, has taken a huge step in adjusting this balance but remains in a risky limbo, with the old forms and personalities – or ‘le pouvoir’ – still haunting the political scene and suppressing the political imagination. But to avoid the disastrous destabilisation of this region, innovative and bold political dispositions will have to be embraced.

Defining the modern Arab state

The two critical challenges for all Arab states are the accommodation of dissent and the management of foreign power, both economic and political. Governments need to set tolerances at levels that enable the best of both political pluralism and foreign partnerships to be integrated into the process of state building. Foreign influence that cedes sovereignty either geographically, ethnically or economically to an external power cannot be part of a long-term sustainable polity, but neither can the exclusion of foreign investment, skills-sharing or defence.

Setting and policing these tolerances will demand a level of statecraft, jurisprudence and procedural integrity that has hitherto not been demanded of Arab governments, where deals between interest groups have replaced institutional representation, and accountability and foreign intervention have too often been accompanied by a compromise of sovereignty.

The voices in the streets of Beirut and Basra are calling time on this form of government. Many of the protesters were not born when the events whose anniversaries are being marked in the Middle East and Europe happened. Their demands articulate the cumulative frustration of a generation at a failure to administer the wealth and talent of a nation. They call for a deep change in what constitutes the modern Arab state.

The Middle East: from regime change to state change, John Raine, the International Institute for Strategic Studies