When Barack Obama took office in January 2009, he invited comparison to Franklin D. Roosevelt, a Democrat who also took office in the midst of an economic crisis. Echoing Roosevelt, Obama promised an initial “hundred days” of achievement, and let it be known that he had been reading Jonathan Alter’s book, The Defining Moment: FDR’s First Hundred Days and the Politics of Hope. Democrats, for their part, hoped that, like Roosevelt’s landslide victory in November 1932, Obama’s victory would give way not only to major legislative accomplishments but to a new long-term Democratic majority.

Four years later, Obama still invites some comparison to Roosevelt. Obama won re-election, and by a sizeable four point margin over challenger Mitt Romney, while Democrats strengthened their hold over the US Senate and won back seats in the House of Representatives. These results suggest that something like a political realignment has taken place. Obama can also boast of passing a health care bill that will make insurance accessible to almost every American, a financial reform bill that will remove some of the temptations that led to the 2008 crash, and a stimulus bill that helped pull the country and the American auto industry out of the worst recession since the 1930s.

But Obama’s achievements have fallen short. After a few initial policy successes, his program of economic and social renewal stalled. Republicans were able to force compromises that actually impeded the economic recovery. They were also able to weaken Obama’s healthcare and financial reform initiatives. And while Obama may be off to a good start in his second term, he is facing the same determined opposition that Roosevelt only had to meet midway through his second term. That suggests a more complicated, thorny political process than that of the Roosevelt realignment.

One way to understand this process is to draw a distinction between the political system that nominates and elects public officials and what political scientist E.E. Schattschneider called the “pressure system” of non-governmental, non-party lobbies; interest groups; think tanks; and political organisations. The two systems are, of course, related; one influences the other. Nonetheless, this distinction makes it possible to understand how a political realignment, like that which led to Democratic victories (in four of the last six presidential elections), has not resulted in unalloyed Democratic legislative accomplishments but instead in something like legislative gridlock.

The theory of political realignment, first formulated in the 1950s by political scientist V.O. Key and later refined by Walter Dean Burnham, holds that American politics goes through cycles of party dominance where one party has generally — although not without interruption — held control of the presidency, Congress, and a majority of statehouses. Republicans were dominant from 1896 until 1932 (although due to a temporary division in Republican ranks, Woodrow Wilson and the Democrats took control from 1912 to 1920); Roosevelt and the Democrats were dominant from 1932 until 1980, when Ronald Reagan won the presidency and Republicans took the Senate.

When a party has become dominant, it is not, however, on behalf of a single interest group or idea. Unlike many European parties, American party coalitions are heterogeneous politically and open to fissure. The coalition partners can disagree with each other over a range of issues, but they tend to agree with each other more than with the party’s opposition. During the New Deal years, Southern whites and Northern blacks both voted Democratic, while disagreeing over civil rights; during the Reagan years, upscale suburban women often disagreed with the social views of right-wing evangelicals, but, along with their husbands, still backed Republican economic and foreign policy.

The parties have platforms, but these are usually ignored by voters. What is important is that the parties uphold a certain world view, or general outlook. Reagan Republicans opposed the welfare state and the taxes they assumed were meant to fund it; they worried about Soviet world domination; upheld the traditional family against the counter culture; and believed that business was hamstrung by regulation and taxes. The Reagan coalition brought together the white South and Northern ethnic suburbanites (products of the white backlash against civil rights), small and large business, and the traditionally Republican farm belt and non-industrial Midwest.

This Republican realignment differed from the Republican realignment of 1896 or Roosevelt’s realignment. It occurred gradually — beginning in 1968, and not really reaching a consummation. By the time Republicans captured the House in 1994, their popularity had begun to ebb. That was, perhaps, because there wasn’t a single cataclysmic event — such as the depression of 1893 or crash of 1929 — that precipitated it and swept aside the opposition party.

The new Democratic realignment began in the 1992 election when Bill Clinton won back the White House after three Republican presidential terms, but like the early conservative realignment, it has suffered several setbacks: in 1994, over Clinton’s political mistakes during his first two years; after September 11, 2001, which revived the public’s support for an aggressive Republican foreign policy; and in 2010, after Obama’s political mistakes during his first two years. But if one were to do a regression analysis of the results from 1992 through this election, one would find a trend away from the Republicans and towards the Democrats.

The new Democratic coalition consists of two groups that used to vote primarily Republican. Professionals (as opposed to managers) were once the most dependable Republican constituency, but they have grown in variety and numbers — from about five per cent of the electorate in the 1950s to about 20 per cent today. Some surgeons and dentists may still vote Republican, but not teachers, nurses, software programmers in Silicon Valley, or entertainment and media professionals. This change is a product of the social revolution of the 1960s, which had its most profound impact on college-educated professionals.

The second key group is women, who, as late as the 1976 election between Jimmy Carter and Gerald Ford, tended to vote for Republicans rather than Democrats. Since then, the “gender gap” has become a key factor in Democratic success. College-educated women were affected by the same forces as professionals, but in addition, many women — and particularly the growing ranks of younger, single women — were offended by the Republican embrace of the religious right.

Two other important groups had voted Democratic since the 1930s, but have now shifted even more dramatically into the Democratic column. Blacks, who had traditionally backed Republicans as the party of Abraham Lincoln, began voting Democratic in the 1930s because of Roosevelt’s support for welfare and unemployment programs, but as late as 1960, Republican Richard Nixon was still getting a third of the black vote. In 1964, the Republican nominee Barry Goldwater opposed the new Civil Rights Act and forged a Republican alliance with the white, Southern proponents of racial segregation. Since then, blacks have become part of the Democratic Party.

Hispanics, or Latinos, also voted Democratic during the New Deal years, but after the Republicans in California began stigmatising Mexican immigrants in the 1990s, they too began to back Democrats by more than two-to-one margins. Fifty years ago, blacks and Hispanics made up less than ten per cent of the electorate. This year, they made up 24 per cent, and, along with Asians, other Democratic-leaning minorities made up 28 per cent of the vote. Support from these groups has, to a great extent, made up for a loss of support among white working class voters who began leaving the party over civil rights in 1968. Outside the Deep South, where whites vote uniformly Republican, Democrats can win elections in many states with 40 to 45 per cent of the white vote.

There is one other constituency that has gone Democratic. In the past, young voters would more or less reflect the prevailing political fashion. Young voters in the Reagan years voted slightly more Republican than Democratic. But beginning with Bill Clinton’s campaign in 1992, the youth vote has increasingly become Democratic — and far more so than any other age group. There have been specific issues — like Republican support for the Iraq War — that drove young voters to the Democrats, but most generally, what seems to have turned them is Republican identification with social conservatism. Republican opposition to gay marriage, for instance, pleased some older, white, rural voters but turned off many of the young.

The world view of this new coalition very much reflects that of the professionals. It is socially liberal on civil rights, women’s equality, gay marriage, and immigration; in favour of government protection of consumers and the environment; wary of big banks and corporations without being anti-capitalist; supportive of free trade; and skeptical about overseas military intervention. These new Democrats back national health insurance; they don’t support pouring money into cities to rebuild them. They want to rein in hedge funds, but they don’t necessarily want to do away with them. They view the Republicans as the party of the selfish rich but lack the populist edge of the older New Deal Democrats. Of course, not all groups within the coalition agree with all of these views. Many African Americans are leery of gay marriage; some labour Democrats would have liked to nationalise the banks and slap tariffs on imports. But the different parts agree more with the professionals’ world view than with the primarily antigovernment world view of the conservative Republicans.

The Democratic coalition was well on display in the 2012 election. In spite of an economy suffering from a 7.9 per cent rate of unemployment, Obama became the first Democrat since Roosevelt to be re-elected with a majority. (Clinton fell short of 50 per cent twice.) The Democrats increased their Senate majority from 53 to 55 even though twice as many Democratic senators were up for re-election (Senators are elected for phased six-year terms.) Democrats actually won the overall House vote but fell short in seats because, after the 2010 elections, Republican governors were able to gerrymander Congressional districts to favour their party.

Exit polls don’t count professionals, but they do count voters with post-graduate degrees, which is a fair approximation. Nationally, Obama won these voters, who made up 18 per cent of the electorate, by 13 percentage points. He won women voters by 11 percentage points; he got 93 per cent of the black vote and 71 per cent of the Hispanic vote; and he won voters from 18 to 29 years old by 23 percentage points. He was able to win from 40 to 45 percent of the white vote in the industrial Midwest and West, and above a majority in the Northeast.

The Republicans have become an overwhelmingly white party based in the Prairie states, the upper Rockies, and the South (excluding Florida and increasingly high-tech upper South states like Virginia and North Carolina). Romney won Mississippi easily by taking 89 per cent of the white vote. But that’s not a formula that can win national elections any longer. Republicans could, perhaps, attribute their defeats in 2008 to George W. Bush’s unpopularity, but they can’t explain away their defeat in 2012, which occurred in an economy that should have favoured the opposition party. By casting their lot after 1968 with the white South and right-wing evangelicals, they have painted themselves into a corner.

Republican setbacks have not, however, entailed dramatic advances for Obama and the Democrats. After Roosevelt took office in 1933, he immediately overhauled the banking and securities industries and set about altering the structure of agriculture and industry. By the end of his first term, he had signed legislation establishing social security, put the power of government behind the right to unionise, and revived the progressive income tax. In his first year, Ronald Reagan transformed the tax code and began to chip away at the welfare state.

Obama started with a similarly ambitious agenda, but he was impeded at almost every turn. Obama’s health care initiative barely passed — and only after being seriously weakened. The administration could have extended Medicare, which is confined to senior citizens, to all Americans. That would have been simpler and more cost-effective and would have benefited American businesses. Instead, the final plan retained insurance companies as expensive intermediaries and was filled with loopholes that allowed their rates to soar.

Similarly, the administration could have revived the New Deal–era Glass-Steagall bill, which separated commercial and investment banking, and which the Clinton administration, under pressure from the banking lobby, had foolishly repealed. Instead, it passed a Byzantine and easily manipulable law that has not prevented further scandal and excess. And faced with a stagnant private sector, the administration could have helped the states maintain public employment until the private sector paid down its debts, but instead it was bullied by the Republicans into forcing the states to cut spending and jobs. Not even Ronald Reagan had done that during the 1982 recession. As a result, the recovery has lagged.

There are several reasons why Obama, even in the midst of a crisis, had to settle for far less ambitious policies than Roosevelt or Reagan. The first is that he failed to rally the country behind his own policies. He mistakenly thought he could win bipartisan support by operating behind the scenes. He eschewed populist appeals and finger-pointing, initially suggesting that both Wall Street and Main Street were to blame for the crash. He was — as befit the change in his party — a cool professional who didn’t like to rail against malefactors. But in the absence of full-throated populist appeal from the President, activists on the Republican Right filled the political vacuum. The Tea Parties emerged as the main grassroots force in American politics sometime during the summer of 2009.

Obama was also frustrated by the separation of powers and divided government that allows opponents in Congress to block White House initiatives. Republicans were able to use a Senate rule requiring a supermajority to end debates to prevent some of Obama’s appointments and initiatives from even coming up for a vote. But that’s only part of the explanation. In the past, the parties had been very reluctant to use this power to filibuster. Under Obama, the Republicans used it repeatedly, even over minor appointments to block what would have been majority votes in favour. That suggests an additional factor is at work. Over the past two decades, the Republicans have become an insurrectionary party that disdains compromise. They used not only the filibuster, but, in the summer of 2011, used what had always been a pro forma debate over raising the federal debt ceiling to demand huge spending cuts from the Obama administration. In employing these tactics, the Republicans defied a long-standing consensus against taking steps that might imperil America’s international standing. Republicans were warned that holding the debt limit hostage could precipitate a drop in America’s triple-A bond rating, but they pressed ahead, determined above all to defeat Obama and the Democrats.

Finally, there is a third factor that is important in explaining Obama’s difficulties in getting his agenda through Congress — indeed, in explaining why some of these initiatives were themselves less than the crisis merited or that Obama himself supported. American political parties have always been somewhat disorganised and decentralised. They have been subject to influence by organised interest groups. During the late 19th century, the railroads controlled many of the state legislatures. In the two decades after World War II, a powerful labour movement squared off against major business groups to shape decisions in Washington. In the 1960s and early 1970s, popular movements, including Ralph Nader’s public interest groups, had much to do with social and regulatory legislation that got adopted. Political scientists described this system of interest group influence as democratic pluralism — a democracy of competing groups rather than individuals.

But in the early ‘70s, in response to the creation of new regulatory agencies and of the threat of an alliance between new left and organised labour, businesses entered politics in earnest. In 1971, only 175 businesses had registered lobbyists in Washington. By 1982, 2,445 had. Businesses established political action committees to influence elections directly. They funded new think tanks and policy groups that, under the cover of scholarly objectivity, published reports and testified on behalf of the businesses that funded them. And they undertook a campaign to prevent the growth of the labour movement and to eliminate unions from existing plants and offices. They were remarkably successful. The American labour movement, which once represented a third of American workers, now represents barely 10 per cent, many of whom work in the public sector.

Business’s influence over American politics has also been reinforced since the 1970s by an alliance with the Republican Party. Groups like the Chamber of Commerce and National Federation of Independent Business, which had once been non-partisan, now give almost exclusively to Republican candidates. In the 1990s, anti-tax activist Grover Norquist, working with top House Republicans, developed a coalition strategy that paired business with conservative social and religious groups. Groups such as the Christian Coalition of America and the National Rifle Association backed business legislation (or joined in blocking Democratic initiatives); in exchange, business either stayed neutral or took the Republican side in battles over guns and abortion. 

Of course, labour unions and groups aligned with the new Democratic Party have lobbies and policy groups of their own. But they are dwarfed by those of business. Labour unions account, for instance, for only one per cent of the money spent on lobbying; and single issue groups, which included conservative as well as liberal organisations, for only five per cent. Their best hope for influence has been through the political system itself, but even candidates backed by labour, environmental, or consumer organisations have fallen under business’s sway once they got into office.

Business influence in the pressure system has been reflected in the legislation passed over the last thirty years. The American tax code has become increasingly regressive. It favours wealth (including unearned interest, dividends and capital gains) over income and business profits over wages. It has contributed to growing deficits and widening inequality, which helped bring about the Great Recession. During Clinton’s presidency, the financial community succeeded in diluting federal regulations — which also contributed to the subsequent crash.

And the list goes on. Telecommunications companies wrote regulations that preserved an anarchic system and have contributed to America’s inferior cell phone network and internet. And in Obama’s first term, business lobbies, working with Republicans, were responsible for skewing the administration’s healthcare and financial bills.

Business, of course, has always played a large role in American politics, but during periods of economic crisis after the depression of the 1890s or during the Great Depression, significant segments of the business community sought out policies that benefited the public interest or business in general rather than particular private interests. During the last thirty years, that has not been the case. Businesses have sought to protect their own without a view to what is generally in the interests of the public and of business.

American corporations, for instance, are at a disadvantage because they have to include rising healthcare costs for their workers within their costs of production. And some big American companies have periodically urged far-reaching health insurance reform. But when it has come to developing policy in organisations like the Business Roundtable or Chamber of Commerce, the companies that stood to benefit from reform deferred to the insurance companies and to other opponents of any government plan. As a result, even under Obama’s plan, businesses will have higher wage and benefit costs than their Canadian, European, or Asian competitors.

Similarly, American industry did not benefit from the repeal of financial regulations during the Clinton years or lax enforcement during George W. Bush’s presidency. But goods producers deferred to the powerful financial services lobby when it came to regulating finance. As a result, the US suffered a financial collapse in 2008 and will likely experience another crash in the decades to come. A far-seeing, less parochial business class might have promoted the kind of regulations that in the 1930s kept the country free of crisis for fifty years. But as business’s power and influence have grown, their vision of what benefits them has shrunk.

Will Obama be able to turn things around in his second term? American presidents usually cannot get much done domestically in their second terms. Many of them, like Reagan or Clinton, focus on foreign policy instead. But Obama can’t avoid domestic economic issues. Even before being sworn in on January 21, he had to negotiate an agreement with Republicans in Congress to prevent automatic budget cuts and tax increases that would have threatened the recovery. And he will face still another challenge over the debt ceiling and budget cuts in the months to come.

In conducting negotiations over what was called the “fiscal cliff”, Obama showed that he learned something from the difficulties he had during his first term about the political and the pressure system. The political system is national and draws upon the support or opposition of individual voters — registered in ballots but also in opinion polls taken between or before elections. The pressure system is headquartered in Washington, and functions best when political decisions are made in private hearings and backroom negotiations.

In his first term, Obama often played into the hands of the pressure system — and of the business lobbies that dominate it. He allowed much of his health care plan to be shaped in White House and Senate negotiations of which the public was barely aware. He took to the hustings on only two occasions over the fourteen months the bill was being debated. In the negotiations over the fiscal cliff, Obama took the issue to the public and framed it in a way that would win national, public support.

As a result, he enacted legislation that eschewed spending cuts and raised taxes on the wealthy, but not on middle or lower income taxpayers. That was a moral victory, but also a victory that will benefit the economy, because the wealthy, who tend to save rather than consume their earnings, will now contribute needed government revenue without imperilling the recovery. Spending cuts or a middle class tax increase could have pushed the United States back into recession.

But Obama’s action was still largely defensive. The American tax code remains dramatically skewed towards income rather than wealth and profit. If not reformed, it will not sustain an ageing population and rising healthcare costs. Reform of this kind, however, will require larger Democratic majorities and a challenge to business’s supremacy and short-sightedness.

American politics does not appear ready for that kind of reform. It might have been possible during the height of the financial crash when the public was ready to accept radical alternatives and when the powerful financial sector was on the defensive, but that opportunity has passed. It might also be possible if the Democrats can score impressive victories in 2014 and 2016, but the party in power rarely does very well in by-elections, and parties have trouble winning three successive presidential elections.

What looks more likely is a continuing alternation between incremental and defensive change, like those that Obama won in January, and grating political conflict that will lead to paralysis and inaction. The American public does not want this kind of gridlock, but the imbalance between the political and pressure system — and the sharp rightward turn of the Republican party — has dictated it.