Seventy-one years ago representatives of Australia, New Zealand, and the United States signed the ANZUS mutual security treaty in San Francisco to reinforce stability in East Asia after the outbreak of the Korean War. Since then the Diggers and GIs have served side-by-side in every major conflict threatening regional and global order. Today the alliance enjoys strong bipartisan support in both capitals as the transitions from Donald Trump to Joe Biden and Scott Morrison to Anthony Albanese demonstrated.

But the reality is that during these seven decades of mateship there were also moments of doubt, risk and divergence. And the coming decades promise to serve up some of the toughest geopolitical challenges yet.

Are we up for it?

History suggests we are. When confronted with new challenges, Australians and Americans have always made the changes needed to recommit to the original premise of ANZUS – that together we could help keep the Pacific secure.

The reality is that during these seven decades of mateship there were also moments of doubt, risk and divergence. And the coming decades promise to serve up some of the toughest geopolitical challenges yet.

At the very beginning of the Cold War, there was divergence on strategy. Australians wanted a firm US commitment to the security of the South Pacific and were still wary of Japan’s re-emergence, but Americans saw Japan as the most important pillar to containing communist expansion and did not want to be distracted from that mission with security commitments in the Pacific Islands.

After Australian and New Zealand troops fought alongside the Americans in Korea, Washington’s attitude changed and negotiations for a security treaty with Australia and New Zealand were launched alongside those underway with Japan. ANZUS was born.

At the beginning of the Vietnam War President Lyndon B. Johnson begged British prime minister Harold Wilson to at least send the famous Black Watch pipes and drums so that average Americans could see that its closest ally was in this fight with us. London refused, but Australia did send troops – indeed, prime minister Robert Menzies had urged Johnson to draw a strong line against communist expansion in Indo-China in the first place. Americans began to appreciate just how close an ally Australia was, but protests against the war also led to new political voices questioning the alliance.

The 1980s saw the second transformation of ANZUS since its creation as the US suspended its treaty obligations to New Zealand rather than submit to Wellington’s nuclear-free zone. The Reagan administration was especially worried that Australia or Japan might follow suit, effectively negating American naval access to the Western Pacific. But after intense fights within the Labor Party, Australia held firm.

The post-Cold War environment of the 1990s tested the alliance in new and different ways as Canberra pushed Washington towards a more active participation in regional multilateral initiatives like APEC and Washington pushed Canberra to take on more military responsibilities in crises like East Timor. There was friction at the outset on both these issues, but US diplomacy proved up to the challenge as did the Australian Defence Force.

September 11, 2001 transformed the alliance yet again. John Howard was in Washington that fateful day and the US had no closer ally than Australia in the subsequent fight against terrorism. For the first time in its history, ANZUS was invoked – but in defence of the US and not of Australia as the architects of the alliance might have expected. Senior Australian officers took up positions within the US chain of command and the levels of intelligence, trade and diplomatic cooperation reached new heights.

But the Iraq War – like Vietnam – also raised dissenting voices on the alliance, particularly as Australia’s economic future appeared more tied to China’s market (despite the underlying reality, it should be noted, that foreign direct investment remained overwhelmingly Trans-Pacific and with the US).

Beijing, it turned out, had different ideas about how it would use its growing power, though. Yes, countries in the region could grow rich off China’s success, but they would also have to abide by China’s rules. Those that did not would face economic embargoes, foreign interference, and military threats. Australia has been at the blunt end of all three tools. The geopolitics of this new era are unmistakable. And so now we enter the newest chapter of the alliance.

In this next chapter, the alliance is looking for big tools of its own. AUKUS will be the most complex fusion of defence industrial technologies by America with an ally since the 1958 transfer of US nuclear propulsion technology to Britain. Less noticed in AUKUS is the advanced capability commitment under which the US, UK and Australia will work together on artificial intelligence, hypersonic propulsion and other emerging technologies.

If America is still the “indispensable power” as the late secretary of state Madeleine Albright liked to say, then it will be critical for its closest allies – and particularly Australia – to understand what makes America tick and how to work across the Pacific to encourage the robust US engagement the region needs.

The US-Japan-India-Australia Quad, for which Prime Minister Anthony Albanese flew to Tokyo in his first week on the job as PM, will reinforce maritime security and provision of public goods such as vaccines while demonstrating the geopolitical consequences of coercion to Beijing. And the new US-led Indo-Pacific Economic Framework has attracted 12 major regional economies despite Beijing’s obvious displeasure with the agreement.

But this new toolkit also raises new questions for the alliance.

How do we work together on advanced technologies and compete against China when so much of technological innovation depends on global cooperation? Can we move from the still vague promise of IPEF to something substantial enough to fill in the void left by the American withdrawal from the Trans-Pacific Partnership? Can the US and Australia eventually return to the vision of the 2007 Sydney APEC Summit for a Free Trade Area of the Asia Pacific that includes China or are we destined to see only further economic decoupling?

Can we formulate diplomatic strategies like AUKUS and the Quad that align the big maritime powers without alienating the smaller powers in Southeast Asia and the Pacific Islands that would prefer to focus on development and climate rather than geopolitics?

And can we continue deepening support for the alliance with a much more diverse generation of younger Americans and Australians who see the value of security cooperation but also want a more comprehensive and progressive agenda for the future?

These are questions that require not only all-of-government solutions but really all-of-society answers.

As I take on the role of CEO of the US Studies Centre, I see the USSC as well positioned to help find those answers. Every year about 1500 University of Sydney students take the USSC courses on American politics, culture and diplomacy so we are well-positioned to understand the aspirations of the generation that will carry this alliance over the next 70 years.

In my earlier work at the Council on Foreign Relations and the Centre for Strategic and International Studies, I saw the potential for independent think tanks like USSC to use their convening power to bring government, business, civil society and academia around the table to chart solutions to problems like technology competition that defy traditional ­bureaucratic or national demarcations.

And finally, if America is still the “indispensable power” as the late secretary of state Madeleine Albright liked to say, then it will be critical for its closest allies – and particularly Australia – to understand what makes America tick and how to work across the Pacific to encourage the robust US engagement the region needs. That, too, is at the core of what the USSC does.

In the 1980s British foreign secretary Lord Carington reportedly listened quietly as his European counterparts complained in a meeting about everything American and then responded, “yes … everything you say is true, but they are the only Americans we have”.

No country in the history of the world has been a greater force for good than the United States despite the many easily identified blemishes and mistakes. The US alliance with Australia has been one important reason for that and Australia’s voice will grow even more important as the geopolitical centre of gravity shifts to the Indo-Pacific. But broadening and deepening the bilateral dialogue will be essential.