What gets measured: Contract performance on the voyage to Botany Bay

In his classic study of Australia's convict system, published more than 15 years ago, Robert Hughes had some unpleasant things to say about the contractors who transported Britain's outcasts to these shores. Hughes contrasted the First Fleet, which he claimed was government-operated and where only a few of the prisoners died, with the Second Fleet, which was contractor-operated and where one in four of the convicts died.¹

For the critics of private provision, the implication was obvious: the private sector was obsessed with commercial returns and could not be trusted to deliver core public services such as the custody of prisoners.² In fact, Hughes was mistaken: all of the convict fleets, including the First Fleet, were contractor-operated. The difference lay in the design of the contracts - the evaluation criteria, the performance regimes, and the contract monitors. The First Fleet cost almost £55,000, a staggering sum that caused a great deal of consternation in Treasury and resulted in the contract for the Second Fleet being let to the lowest bidder, at less than half that cost. No one seems to have particularly cared about the background or the reputation of the winning contractor, and it turned out that the lowest bidder was a slave-trader, with a great deal of experience in transporting prisoners, although under very different conditions.

The contract monitor on the First Fleet was Arthur Phillip, who refused to let the ships leave Portsmouth until adequate food and provisions were on board, and insisted that they stop three times on the voyage out, so that the prisoners could stretch their legs. By contrast, the contract monitor on the Second Fleet was a junior official, "an ineffectual hack of questionable competence" (as one historian has described him). He ignored concerns about the quality of the food raised by the ships' surgeons and he was plied with large quantities of alcohol by the ships' captain for much of the voyage.

The performance regime in the contract for the Second Fleet included extremely heavy penalties for escapes, so that the contractor kept the prisoners in heavy slave shackles while they were in port. This had an impact on their health before they had even left Portsmouth, and it created a strong incentive not to stop too many times on the way out. Moreover, the contractors were allowed to sell any unused food and provisions when the ships arrived in Port Jackson and later in Asia (and as it turned out, they made a fortune on these sales).

Of course, this created a perverse incentive for the contractors. As one of the military officers noted during the voyage: "The more they can withhold from the unhappy wretches, the more provisions they have to dispose of at a foreign market."³

The death rate on the Second Fleet caused a scandal in Whitehall, and the service reverted to the original contractor. The contracts were amended to impose a financial penalty for deaths in custody and the ships' surgeons were appointed as contract monitors, with detailed reporting on the prisoners' well-being.⁴

So while my wife's great great grandfather was lucky to survive the Second Fleet, by the time my own great great grandfather was shipped out in the 1840s, he had a relatively easy passage.

It is difficult to think of a more dramatic illustration of the maxim, "What gets measured gets done". And in contracting for public services - where so much depends on the evaluation criteria by which the service provider is selected, and the performance criteria through which it is rewarded - what we measure lies at the heart of whether or not we serve the public interest.

For much of its history, competitive tendering and contracting has been used by government as an instrument for driving down costs. And the evidence of the past two or three decades confirms that it is highly effective at doing just that. However, for reasons that are difficult to explain there has been a reluctance to acknowledge that competition and contracting can also be used as a means of driving through service improvements.

We know that it can be used for this purpose, since in the UK we have several rather impressive examples:

Prisons Management

In the case of prisons management, the Home Office seized the opportunity created by competitive tendering to demand higher standards of the private contractors - much longer hours out of cell, increased levels of purposeful activity and so on.

The result was that the private prisons have made a major contribution to the so-called "decency agenda". From the very first contract, prisoners were called by their first names, custody officers wore name tags and were likewise called by their first names, prisoners were given a key to their own cells, and significant numbers of female custody officers were introduced into the prisons for the first time, bringing down the testosterone levels overall.

Critics of The Wolds, the first privately managed prison in the UK, were silenced when prisoners began to speak out on behalf of the contractors. John, 31 years old and awaiting trial on a shooting charge, told the press: "This is the best prison I've ever been in, and I've been in prisons all my life. It's more humane. It's not uptight. The guards are polite."⁵

Prisoners didn't care who managed the prisons, as long as they were treated with respect. One of them wrote to the Observer newspaper shortly after The Wolds was opened, rejecting the philosophical arguments raised by prison reformers:

As someone who is committed to penal reform and as a prisoner, I prefer to adopt a more pragmatic approach to this issue. Today I will spend 18 hours locked in my cell and I will spend tomorrow in exactly the same way. I look with envy at the Home Office tender document for the Wolds privatised remand centre near Hull, which demanded the delivery of a regime guaranteeing a minimum of 12 hours per day out of cell. When Group 4 signed the final document, that figure had been revised upwards to 14 hours.⁶

The UK Prisons and Probation Minister, Paul Goggins, has now announced the government's intention to market-test the management of three existing prisons, with the stated intention of "driving up standards and encouraging imaginative approaches to managing offenders in custody".⁷

Failing LEAs

The pioneer in the use of competition and contracting to turn around poorly-performing public services was the Department for Education and Skills. In 1999 it opened up the management of nine failing local education authorities (LEAs) to management by the private sector.

For those unfamiliar with the British education system, an LEA is roughly the equivalent of a region of a state department of education in this country, each responsible for somewhere around 100 schools and managed by a local authority.

The UK government had introduced a program of LEA inspections using Ofsted, the Office of Standards in Education, with public reporting of the results and the threat of intervention in those LEAs that proved incapable of reforming themselves.

Of 20 interventions, 11 involved a "fresh start" by the Department for Education, with the replacement of management and teachers; in the other nine, the management of the entire LEA was thrown open to competition.

The Confederation of British Industry has just published the first comprehensive study of the results of this experiment. It has found that, on the key performance indicators:

• the nine privately managed LEAs have improved more than the average for all LEAs in England;
• they have performed better than the 11 'fresh start' interventions where competition and contracting were not employed; and
• they have improved more than the average LEA with the same low level of attainment in 2000.⁸

Contracting for what?

So we know that competition and contracting can be used to improve the quality of public services. But for the most part, this is not how the private sector has been employed.

In fact, in the 1980s, when governments in the English-speaking world turned once again to contracting for the provision of public services, there was insufficient recognition of the need to specify service standards carefully and monitor the contractor's performance to ensure that standards were maintained. The result was that, in too many cases, contracting came to be associated with a deterioration of service standards, bringing discredit to the entire concept of private provision.

At the same time, some governments deliberately used competition to drive down the terms and conditions of public service workers, causing many people in the community (and unions in particular) to conclude that contracting was invariably associated with poor workplace relations. In fact, at the same time as they were using contractors as shock troops to bring down public sector wages, some governments were also criticising the contractors for their poor employment practices.

And in the UK, where public-private partnerships have been used (to some extent) to shift infrastructure spending off balance sheet, government agencies have been given the incentive to maximise risk transfer, rather than allocating risks sensibly to the parties best able to manage them. The result has been that some of the government's most high-profile procurements have been responsible for generating bid fever among bidders, so that the winners are almost certain to fail.

Lest I am misunderstood, let me make it clear that I am not suggesting that competition cannot be used in the ongoing pursuit of value for money. Both in prisons and LEAs, competition and contracting have delivered significant and demonstrable value-for-money improvements. We don't have the aggregate numbers for the LEAs, but in prisons, the average cost savings are in the order of 20 per cent.

But value-for-money is not the same as crude cost-cutting, where the savings are delivered by making employees and end-users pay. One of the implications of using competition in this way is that the public loses confidence that contracting can be used to deliver core public services. I have been told on a number of occasions by intelligent and fair-minded union officials that "the only way that you are can achieve these savings is by cutting workers' terms and conditions".

In the UK, there is clear evidence that the private sector has grown tired of being associated with poor service delivery and poor employment practices. In the past month, as a result of a television campaign by the "Naked Chef", Jamie Oliver, a number of leading catering contractors have made it clear that they are not interested in trying to deliver nutritious and edible school dinners for 37p a meal.⁹

Over the past few months, the leadership of Unison, a major public sector union, has been conducting a highly political campaign over the cleanliness of hospitals that are cleaned under contract. In fact, the evidence indicates that contracted hospitals have a small (but statistically insignificant) edge over hospitals with inhouse providers in terms of cleanliness - in essence, there is no difference between the two groups. But I am aware of one provider that has begun a campaign to expose the fact that the UK National Health Service pays one-quarter of what the private sector pays for equivalent cleaning services.

Quite apart from reputational impacts, there is some evidence to suggest that contracting for service improvement is important in motivating front-line staff, particularly where they have transferred from the public sector. The Serco Institute has recently published a paper in which we looked at competition and contracting through the eyes of front-line service managers.¹⁰

We interviewed 13 of our contract managers who used to deliver the same services inside government, and we asked them what is the difference between managing front-line services under the bureaucratic and contract models? I won't go into the findings of that research here, but one of the many issues that we probed was how public service managers accommodate the company's need to make a profit.

Our research suggests that it is extremely important to those responsible for actual delivery that they are able to deliver high quality public services. As one of them told us:

I have no qualms about delivering a profit to shareholders from a public service. I see no conflict in this, because what I deliver is better value for money for the taxpayer and higher standards for the people who use the service. So it is a win-win-win situation.

Indeed, from where these people stood, the aggressive pursuit of cost savings by the government could be just as threatening to their ability to deliver high-quality public services as an overly commercial approach on the part of the company.

In the debate over the so-called "public service ethos", I have begun to argue that if it is allowed to operate unchecked, the "savings imperative" of government can be just as destructive of this culture of public service as the "profit motive" of the private sector.

Besides the possibility that they just don't care - which I will park for the moment - there are a number of reasons why public officials fail to address these qualitative issues. They can be broken down into those three factors that I identified earlier in talking about the Second Fleet: the evaluation criteria, the performance regimes and the contract monitors. Let me deal with each of those in turn.

What gets evaluated?

Those who are regularly engaged in bidding for public services know that even when there are strong commitments to the contrary, public officials are inclined to revert to price-based competitions. The explanation that is usually given for this behaviour, in this country and in the UK, is fear of the auditor-general.

And yet in the UK, the National Audit Office has made it clear on a number of occasions that it looks at value-for-money and not just price, and that (for example) a decision not to award a contract to the lowest bidder, in the interest of building a deeper market, is entirely defensible. A better answer, perhaps, is fear of the Public Accounts Committee, where political judgements are made in the goldfish bowl that is created by intense media scrutiny.

But the underlying reason why public officials feel safe with price-based criteria when explaining themselves in front of the Public Accounts Committee is the lack of sufficiently robust qualitative measures. It's easier to prove that you did the right thing when you award the contract to the lowest bidder.

This is an area where government and the private sector need to undertake some joint research in developing a professional literature, looking at procurement processes in general, and evaluation criteria in particular, from both sides. There is a worrying lack of transparency in the process of commissioning and actually procuring public goods and services, and I am afraid that the answer does not lie in appointing so-called probity auditors, who themselves face perverse incentives.

What gets measured?

The second stage at which performance can be controlled is in the design and management of contractual performance criteria. A contract for public services is an agreement which specifies the services that will be delivered by the contractor and the payments that will be made by the customer in return for performance.

It follows that the performance regime lies at the very heart of the contract. The design of those measures, and the financial and reputational rewards and sanctions associated with them, drive service delivery. And it is essential that they are closely aligned with the customer's key objectives.

In light of this, it is disappointing how little the effectiveness of different performance regimes have been studied in practice. What little expertise there is in this area exists in the personal experience of individual consultants. With this in mind, I asked one of my staff at the Institute to undertake a study of around 40 of Serco's public service contracts, in order to understand what works (and what doesn't) in performance contracting, what kind of measuring and monitoring tools are being used, and how they are evolving over time. The research will be published later in the year.

This is not the occasion to spell out the detailed findings, but let me walk you through some of the high-level conclusions.

Performance measurement is not a certificate of compliance

Performance measurement is a tool for delivery, not a certificate of compliance. It is capable of driving results, facilitating improvements and securing desired outcomes. But it will achieve little if it is used merely for ticking boxes and allocating blame.

Successful public services are flexible and dynamic, which means that successful performance measures cannot be prescriptive. There must be room for the service provider to innovate, to adapt as priorities change and lessons are learned.

Of course, there must be consequences for failure, but it is also important that performance targets are managed intelligently and constructively. Performance measurement is meant to deliver service improvement, not schadenfreude.

Ownership rests with the purchaser

Those who lay down the policy outcomes and the performance measures need to accept that they are the ultimate owners of the regime, and the way in which performance is managed. They must understand enough about the contract and the service model so that they can amend the regime if it is contributing to poor performance. And they should take a reasonable and constructive approach to performance assessment to ensure the best overall outcomes.

As contracts become longer, larger and more complex, the likelihood of change during the lifetime of the contract is inevitable. Change is good because it signals dynamism, improvement and innovation. But to enable change, appropriate structures and relationships must be in place, and the contract models need to be constructed with flexibility built in.

There are limits to how much risk can be transferred, particularly in long-term service contracts. Commissioning authorities need to have realistic expectations, otherwise they may drive up the cost of services unnecessarily, failing to deliver improvements in service delivery and inflicting reputation damage on both sides.

As to whether a service is contracted wholly or in part, ultimate ownership for the delivery of that service remains with the commissioner. The commissioner specifies the service that it requires. It chooses the organisation that it wants to deliver that service. It must ensure that the desired service and performance standards are actually delivered.

Performance measurement is only part of the performance regime

Performance management lies at the heart of what we do. We are paid to deliver a service to specified standards. But it doesn't all turn on the contractual performance measures. Beyond the contract we are audited, monitored and inspected by clients, by independent regulatory bodies and by financial institutions.

We are also scrutinised in parliamentary committees, meetings of local authorities, and the media, and increasingly by end-users who tell us what they think through customer surveys, comments and complaints. If we fail to deliver, there are numerous means of redress other than withholding financial payments. In the case of extreme failure, the customer can intervene to terminate the contract, with the potential for reputation damage.

But we also face qualitative audits in a number of sectors, which are not part of the formal contractual process. For example, in the custodial sector, we are subject to audit by the Chief Inspector of Prisons, whose reports have no direct impact in terms of financial penalties, but which are, nevertheless, taken very seriously because of their reputational impact.

It is possible for customers to use reputation to create positive incentives and thereby get a great deal more out of the service provider than the performance targets require. At Docklands Light Rail, the mass transit system which we operate out to London's new financial district, we ensure that the stations are clean and graffiti-free, even though there are no such performance measures in our contract. Like Sherlock Holmes' dog that didn't bark, this is a mystery that deserves an explanation.

Why would a contractor do something that the performance regime doesn't specify? In simple terms, reputation. Docklands Light Rail has won national awards for its performance and it is in our interest to ensure that it retains its reputation for excellent service.

Managing networks

The final stage at which the customer has an opportunity to exercise some control over the quality of service delivery is in the monitoring and management of the contract. Again, this is one of those disciplines that government has tended to overlook. The obsession with procurement and outsourcing has led us to overlook the importance of ongoing management of the relationship.

There is a natural inclination - on the part of the private sector, it seems, as much as the public sector - to assume that once services have been contracted out, they can be managed simply through light-handed monitoring and the imposition of financial penalties. Last month, the Serco Institute hosted a seminar with Danny Ertel, a former Harvard academic who specialises in contract negotiation and management. Danny was the author of a recent article in the Harvard Business Review entitled "Getting past yes: Negotiating as if implementation mattered".¹¹

Danny concentrates on private sector outsourcing in North America, so it was interesting to compare his experiences with ours in the UK public sector. He spoke of a large North American utility where the CEO had presided over the outsourcing of corporate services, with a contract worth several hundred million dollars. Having signed the contract, the CEO was proposing to appoint a single contract monitor. They were able to convince him of the importance of managing this contract over time, and the number of contract management staff was subsequently increased to nine.

It appears that there is a mind-set that accompanies competition and contracting - in North America as much as in the UK and Australia, in the private sector as much as in the public sector - that leads us to believe that once we have signed a contract our responsibility for the service is at an end.

One of the best expositions of this perspective is found in a recent book by Stephen Goldsmith and Bill Eggers, entitled Governing by Network. Using the concept of the network, rather than that of the contract, Goldsmith and Eggers focus on the commissioner's ongoing responsibility for service delivery.¹²

The customer, not the contractor, is responsible for network design. The customer, not the contractor, is responsible for network management. At one level this is a shift in language, but I see it as much more fundamental than that - it is about changing the whole way in which we view this process of competition and contracting.

Believing in the things we do

There is little new about competition and contracting for public services. For 200 years, up until the middle of the 20th century, the private provision of public services was extremely commonplace in the English-speaking world. There are different explanations as to why that system of public management passed away, but after a great deal of close analysis I have come to the conclusion that the bureaucratic organisation triumphed because it was seen as more responsive and more accountable.

There is nothing inevitable about a mixed economy in public services. There is no particular reason why this cycle of public service contracting - which for Serco started more than 40 years ago - needs to continue for another 40 years. It all depends on how well companies like Serco, our competitors and our public sector customers cope with this particular tool of management.

In writing about the private provision of public services recently, the former Labour politician, Roy Hattersley, quoted Gladstone on his resignation of the premiership: "Things are done best by those who believe in them." I agree with Hattersley. But I can't seen any reason why, given the right incentives, private providers can't believe in public services just as much as government providers.

References

1. Robert Hughes, The Fatal Shore, London: Collins Harvill, 1987, pp.144-5.
2. John Quiggin, 'Are there core functions of government?', paper presented to 'Future Directions in Australian Government', IIR Conferences, Canberra, 25 February 1998, p. 5. The great advocate of competition and contracting, Simon Domberger, also made the mistake of accepting Hughes' assertions in Simon Domberger, The Contracting Organization, Oxford University Press, 1998, pp. 8, 10.
3. Michael Flynn, The Second Fleet, Sydney: Library of Australian History, 1993.
4. 'Report from the Select Committee on Transportation', 10 July 1812, p. 10.
5. 'Inmates find good life in Britain's first private prison', Agence France-Presse, 7 February 1993.
6. Mark Leech, 'Please privatise my prison', Observer, 9 August 1992, p. 18.
7. Home Office, 'Raising standards through contestablity: Competition to operate three prisons announced', Press Release, 22 March 2005.
8. Confederation of British Industry, The Business of Education Improvement: Raising LEA Performance Through Competition, London: CBI, February 2005.
9. A week after this speech was delivered, the UK government announced it was injecting another £280m into school meals, bringing the average cost to around 50p. Tania Branigan et al, 'Kelly passes school dinner test', Guardian, 31 March 2005, p. 1.
10. Emma Reddington, Good People, Good Systems, London: The Serco Institute, 2004.
11. Danny Ertel, 'Getting past yes: Negotiating as if implementation mattered', Harvard Business Review, November 2004, pp.60-8.
12. Stephen Goldsmith and William D. Eggers, Governing by Network: The New Shape of the Public Sector, Washington, DC: Brookings Institution, 2004.