Before moving to Yale and becoming a bestselling historian, Paul Kennedy grew up on Tyneside in the 50s and 60s. "A world of great noise and much dirt," is how he remembers it, where the chief industry was building ships and his father and uncles were boilermakers in Wallsend. Last year the academic gave a lecture that reminisced a little about those days.

"There was a deep satisfaction about making things," he said. "A deep satisfaction among all of those that had supplied the services, whether it was the local bankers with credit; whether it was the local design firms. When a ship was launched at [Newcastle firm] Swan Hunter all the kids at the local school went to see the thing our fathers had put together and when we looked down from the cross-wired fence, tried to find Uncle Mick, Uncle Jim or your dad, this notion of an integrated, productive community was quite astonishing."

Wandering around Wallsend a couple of weeks ago, I didn't spot any ships being launched, or even built. The giant yard Kennedy mentioned, Swan Hunter, shut a few years back, leaving acres of muddy wasteland that still haven't lured a buyer.

You still find industrial estates, of course, and they look the part: overalled men milling about, passing lorries. Only up close does it become clear that there's not much actual industry going on.

The biggest unit on one estate is a dry cleaner; on another, a warehouse for loft insulation dwarfs all else. At a rare actual manufacturing firm, the director, Tom Clark, takes me out to the edge of the Tyne, centre of the industrial excitement remembered by Kennedy. "Get past us and there's nothing actually being made for miles," he says, and points down the still waterfront.

At his firm, Pearson Engineering, Clark introduces me to a plater called Billy Day. Now 51, he began at the firm at 16. His 23-year-old son William is still out of work, despite applying to dozens of small factories. As the local industry's gone, so too have the apprenticeships and jobs. "No wonder you get young kids hanging out doing whatever," says Day. "We've lost a whole generation."

You can see similar estates and hear similar tales across the country, from the north-west down to the Midlands and the old industrial parts of suburban London. But it's in the north-east, the former home of coal, steel, ships and not a lot else, that you see this unyielding decline at its most concentrated. It's a process I've come to think of as the de-industrial revolution, in which previously productive regions and classes are cast adrift.

It gets little airtime nowadays, but this story cuts across most of the key debates in British economics: from why the wealth gap has widened so dramatically to why the country remains stuck in a slump. The dismal unemployment figures that you saw on Nov 16 – they're part of it, too. And as David Cameron and Vince Cable this month tout their get-behind-British-manufacturing campaign, what they fail to mention is the bag of bones that the sector has become.

What's driven the de-industrial revolution? In significant part, it's a tale about where Britain is going, one that's been told by Conservative and Labour alike over the past 30 years. It's a simple message that comes in three parts. One, the old days of heavy industry are gone for good. The future lies in working with our brains, not our hands. Two, the job of government in economic policy is simply to get out of the way. Oh, and finally, we need to fling open our markets to trade with other countries because, despite the evidence of countless Wimbledons and World Cups, the Westminster elite believe that the British can always take on the competition and win.

Yet there's ample evidence that the promised rewards of this post-industrial future haven't materialised. What was sold as economic modernisation has led to industrial decay, with too often nothing to replace it.

But before coming to the results in the north-east and elsewhere, let's set out the promises made by the politicians. Over the past 30 years, there have been three main versions of the de-industrial revolution. I call them the Thatcher argument, the Blair vision and the Cameron update. I'll come back to the coalition and the future at the end, but let's start with Mrs T.

By the mid-70s, the press, politicians and academics agreed that Britain was in crisis. And as far as correcting the critical weakness of the British economy went, the Thatcherites had a clear answer. In a word: competition.

In 1974, Keith Joseph – the man Margaret Thatcher described as her closest political friend – gave a speech in which the key section was titled "Growth Means Change". He argued that British industry was "overmanned" with "too low earnings and too little profit and too little investment". The answer lay in shedding factory workers, which would make industrial companies leaner and free up labour for new businesses.

"This is growth," Joseph said. "Whether the new work is in industry, commerce or services, public or private ... The working population must choose between narrow illusory job security in one place propped up by public funds or the real job security based on a prosperous dynamic economy."

Five years later, the Conservatives encouraged just that process: first came an austerity programme that saw nearly one in four of all manufacturing jobs disappear within Thatcher's first term. Then followed privatisations and an economic policy geared towards a housing boom and the City. Despite Joseph's assertions, the middle-aged engineers who were laid off didn't go away and become software engineers – they largely landed up in worse jobs or on the scrapheap.

Workers head home after their shift at a Newcastle shipyard in the 80s. Photograph: Alamy

But it was with Tony Blair that the argument for moving from industry to services shifted from one of dire necessity to being an altogether more optimistic vision about Britain's place in the world. The architects of New Labour were convinced that the future lay in what they called the "knowledge economy". Mandelson declared Silicon Valley his "inspiration"; Brown swore he would make Britain e-commerce capital of the world within three years.

Again, the theme was simple: most of what could be manufactured could be done so more cheaply elsewhere. The future lay in coming up with the ideas, the software, and most of all, the brands. Once the British had sold cars and ships to the rest of the world; now they could flog culture and tourism and Lara Croft.

The odd thing is that all this techno-utopianism came from men who would struggle to order a book off Amazon. Alistair Campbell tells a story about how Blair got his first-ever mobile phone after stepping down as prime minister in 2007. His first text to Campbell read: "This is amazing, you can send words on a phone."

But Blair and Brown had plenty of advisers and consultants and thinktankers to help them stay cutting-edge. One of the most interesting walk-ons was played by an American academic called Richard Florida, who comes up time and again in this saga. Florida argued that the successful regions of the future would be driven by what he called a "creative class" of young people living in city centres. And there was an even groovier elite termed the "super-creative core".

It was always a daft argument: ask Florida who was in this "super-creative core" and he'd suggest all sorts of occupations that didn't sound especially groovy, such as IT support staff.

But what really stuck out was how Florida fenced off creative work. You were either a knowledge worker or a factory worker – as if the other stuff didn't require brains. And running through a lot of the knowledge-economy talk was a carelessness, bordering on contempt, for what people did. That was echoed in the stark choice that Labour presented workers in this entrepreneurial, innovative young country: brain up, or get written off.

There is a popular argument which holds that all the rot in the British economy began in 1979. I don't believe that. Nor am I spinning a tale of leonine industrialists being led by Westminster donkeys. Pearson Engineering's Tom Clark has a good story about how the firm's previous owners used to handle industrial relations. Every time the workforce went on strike, which was often, one of the Pearsons would buy a new Rolls-Royce Silver Cloud and drive through the picket line, waving two fingers at his own staff.

The real charge against Blair and Brown is that, rather than focus on this problem of underpeforming managers and shareholders, they chased the fantasy of the knowledge economy. In their wake trailed a whole phalanx of propagandists.

Take Tyneside's regional development agency, One North East. Its website features only one document on manufacturing, but plenty with titles such as "Towards an E-region". And there is a discussion paper called "The North East: Bohemian or Behemoth?". It highlights a pensioners' day centre, where young and old can chat "on the complexities of digital art". There's also a scheme in Durham that helps young people "avoid the temptations of crime by encouraging them to take up fishing".

What's all this civil servant away-day speak got to do with regional development? All becomes clear with the RDA's discussion of how it can keep "bright, innovative people, with transferable skills" in the area. So that's what it's about: attracting the super-creatives. It comes as no surprise to find that the leaflet was produced for a visit by Florida.

Meanwhile Britain has been undergoing one of the biggest industrial declines seen in postwar western Europe. When Thatcher came to power, manufacturing accounted for almost 30% of Britain's national income and employed 6.8 million people. By the time Brown left Downing Street last May, it was down to just over 11% of the economy, with a workforce of 2.5 million. (Two caveats need to be made. First, manufacturing is partly a productivity game: you get more machines in, so you employ fewer staff on a particular task. Second, other countries have stepped back a bit from manufacturing – all those new Labour-isms about the competitive threat from China and India were not just babble.)

Even so, by any standards these numbers represent a collapse. As the government itself admits, no other major economy has been through our scale of de-industrialisation. The Germans and French have kept their big domestic brand names – the Mercedes and Mieles, the Renaults and Peugeots – and with them their supply chains of smaller suppliers and partners. In Britain there's been no such industrial husbandry, with the result that we have few big manufacturers left – but a profusion of bit-part makers. Is that a bad thing? Plenty of evidence suggests so. Bad economically, and terrible socially and culturally.

You can sum up the economic problem in a word: Greece. Not my comparison, but the one I repeatedly heard in Newcastle. To me, it still sounds too extreme but I saw their point: the loss of manufacturing means Britain no longer pays its way in the world. Last year, we British bought £97bn more in goods from other countries than we sold to them – the biggest shortfall since 1980.

The de-industrialists in Whitehall have long argued that this doesn't matter: that Britain can simply borrow more and sell its assets to foreigners. But there are problems with relying on foreigners for hard cash; they can simply refuse to extend it to you – just ask George Papandreou.

In the north-east, manufacturing jobs have nearly halved since 1997 alone – one of the biggest drops anywhere in the country. So what's come along in its place? The simple answer is: not a lot. A few minute's walk from Newcastle train station is the old Scottish & Newcastle brewery, which is now called Science City. It was meant to be home to hi-tech new businesses, but all you can see there is some fancy student accommodation and acres of barren ground.

Even the good-news stories of de-industrialisation turn out to be pretty grim. In 2005, MG Rover shut its plant at Longbridge in the Midlands. Around 6,300 staff lost their jobs but, the argument ran, if anyone was going to bounce back it would be these skilled staff at a prestige name.

Three academics tracked what happened to 300 MG workers, interviewing them regularly for three years. Sure enough, about 90% got another job. A lot retrained and some went into the service sector. In other words, they did everything the government told them to. Only now they were earning an average of £5,640 less every year than they had at MG Rover. And a quarter of those interviewed admitted to living off their savings or being in financial difficulties.

It's a similar story at regional level. Look for private-sector growth in the north-east and you get the odd high-skilled niche such as computer games in Middlesbrough – but they're never going to provide volume employment.

At the other end of the labour market the north-east is now in with a shot at becoming call-centre capital of Britain. There's Northern Rock, of course, which was a great success story until it collapsed. Finally, you get the public sector, covering up for the weakness of private industry.

The Centre for Research on Socio-Cultural Change at Manchester University calculates that, between 1998 and 2007, the bulk of the new jobs in the Midlands, the north, Wales and Scotland came from the state. And, of course, there's welfare: more than one in six of all people living in the north-east claim some form of out-of-work benefit.

Now think about the cost to the places that used to be home to all this now-rotting industry. Everywhere in Tyneside are traces of its industrial past. There's Newcastle university, founded by the arms-maker William Armstrong, and the local tech colleges, which used to provide training to factory staff on day release. Then there's the literary and philosophical society, the mining and engineers institutes, the social clubs. The culture here was traditionally a productive one rooted in making things and selling them. What the de-industrial revolution has obliged the north-east and others to do is to adopt a consumerist culture – to buy things instead, often on tick.

So it is that you get a giant shopping centre such as Liverpool One describing itself as the largest urban regeneration project in Europe without any apparent irony. Or you hear American economists arguing that free trade may have reduced salaries for blue collar workers in the west, but they can now buy cheaper Chinese imports. In other words, you may have lost your factory workshop – but at least you've got a pound shop.

If you're employed in the service sector or, more to the point, you're a politician who considers that Britain's future lies in services rather than manufacturing, there might seem nothing wrong with that. Still, the net result has been to undermine whatever economic or political clout the old industrial regions and classes had, by making them dependent on Westminster for jobs and welfare.

But, you might say, isn't David Cameron going to change all that? After all this is the government that talks about the March of the Makers. Obviously, it's difficult to disagree with the argument that the coalition put about – that Britain's economy is lopsided, dependent on the City and the housing bubble – but there aren't the policies to go with it.

Instead, Cameron issues the same prescription as Thatcher – that if you cut back on public spending, private spending will inevitably grow. His ministers give train contracts to German factories rather than to workers in Derby. On taking office, the heir to Blair even let it be known that he had a new guru – none other than Florida.

And yet many of the arguments that preoccupy the British are haunted by the spectre of manufacturing. Angry at the overweening power of banks? Then you want a more mixed economy. Distressed at the gap between the rich and the rest of society? In the end, that will require jobs with decent wages and skill-levels, like the old manufacturing jobs.

This applies to non-economic debates, too. Politicians go on about localism, without discussing what de-industrialisation has done to local economies. Pundits bemoan the loss of community spirit without considering the wrecking ball that has been put through many communities.

In Walker, on the Tyne, the workers at Pearson Engineering are recalling when their firm employed 1,000, rather than a couple of hundred. Clark, the apprentice who worked his way up to company director, sketches out his retirement plans. Then he pauses. "I'm beginning to worry about what my grandkids are going to do," he says.

Aditya Chakrabortty's talk on the de-industrial revolution is on Radio 3, 18 November, 10pm.