Robodebt: Reform and Algorithms

2023-08-29

The injustice of debt collection from welfare recipients via Robodebt has been exposed by a Royal Commission. But where to from here in pursuing accountability and preventing reoccurrence?

By Mark Brogan, Volunteer with The Greens (WA). Formerly ECU. Advocates on environment and heritage.

Of bones and Robodebt

As a tool for getting at the truth and reform, Royal Commissions over the years have spawned more detractors than advocates. The English humourist, Parliamentarian and law reform activist, Sir Alan Herbert, was a noted sceptic observing:

“A Royal Commission is generally appointed, not so much for digging up the truth, as for digging in it”,

and:

“appointing a Royal Commission is like a dog burying a bone, except that the dog does eventually return to the bone.” [1]

With no shortage of Royal Commission detractors and many buried, never to be unearthed bones, the persistent popularity of Royal Commissions and the aura of importance that surrounds them seems strange. But popular they remain – at the time of writing no less than four Royal Commissions are in progress Federally or have recently concluded – The Disability Royal Commission, The Age Care Royal Commission, The Defence and Veteran Suicide Royal Commission and of course the Robodebt Royal Commission. Robodebt is easily the most celebrated of the four. 

Robodebt was approved by Cabinet when Scott Morrison was Minister for Social Security in 2015 and was rolled out with its online platform in September 2016. It continued in various iterations for four years.[2] It was expected to recover welfare savings to the government of $1.7 billion over five years, through the detection of customer fraud and overpayment. Replacing manual income auditing with an automated decision-making system and an income averaging algorithm, Robodebt involved a major re-working of social welfare debt recovery. It relied on PAYG annual income data sourced from the ATO to calculate welfare recipient income and compared this with what income recipients were allowed to receive for the income benefit paid. In the event of a discrepancy involving apparent overpayment, a recipient’s income support level could be adjusted, or a debt raised against the recipient. 

The debt calculation relied on an averaging algorithm to translate annualized PAYG data into fortnightly estimates. There were many flaws in the system that made the process of debt recovery both cruel and unfair. But this was the critical one. No warrant existed in social security law that entitled the administering agency, the Department of Human Services, to use income averaging as a proxy for actual income in fortnightly measurement periods. This was known because of legal advice received by the Department of Social Security in 2014. But the savings were simply too tempting. A pilot study conducted in 2014, had shown that debts raised against recipients using the income averaging algorithm and automated decision making were thirteen percent higher, than debts calculated manually. Across a welfare recipient customer base of 867,000, savings of $1.5 billion were suggested.[3]  

Over its lifetime and several iterations, Robodebt ended up asserting debts of $1.7 billion against 453,000 Australians.[4] At its peak in 2017, 20,000 debt notices were being issued per week, with no explanation of how debts were calculated. Because an incoming averaging algorithm had been used as part of automated decision making, many debt notices were wrong. In a population of 453,000 affected welfare recipients, media reports attest to mental health issues, relationship breakdowns and suicides attributed wholly or in part to Robodebt.[5] Far from making savings, after refunds of wrongly calculated debts and other expenses, the Royal Commission concluded that the net cost of the Robodebt scheme to the Commonwealth was $565 million.[6] What had been cast to produce major savings to the budget bottom line, ended up costing tax payers. 

The scandal prompted the establishment of the Robodebt Royal Commission under retired QLD Chief Justice, Catherine Holmes. Over its short life from late 2022 to early 2023, the Robodebt Royal Commission hearings served up a live feed of squirming politicians and bureaucrats as they came under relentless questioning from a talented bunch of KCs. For a while, it was the biggest show in town and a national obsession. There are no prizes for understanding why. Who could resist the breathtaking audacity of what the Morrison Government was up to? Or the fearless and unflappable Catherine Holmes? Or the apparent heroism of Centrelink employees who dared to challenge the narrative of management? Or the heart-rending tales from victims? These stories are important if you want to get your head around the real dimension of harm caused by this illegal system of debt recovery. And why the delivery of justice is so important to its victims.

But now that the Commission is done, and the live feed is no more, what legacy will the Robodebt Royal Commission leave? Has it dug into the truth only to bury it forever like a lost bone as Herbert predicted, or will there be a tangible legacy that promises better government? 

Legacies are hard things to measure because legacies require time. Law reforms and prosecutions do not happen overnight. And legacies are not a natural focus of attention. With the relentless churn of the news cycle, the media is focussed on the here and now. In the case of Robodebt these were the ‘gotcha moments’ where politicians and bureaucrats squirmed under questioning from KCs. In the aftermath, they are the identities of the bureaucrats and politicians singled out for civil and criminal action in the secret appendix to the Report. The 'gotcha moments' have also been the moments of revelation when the shocking truth of how cruel and indifferent our welfare system had become was brought into plain sight. 

But for the media, the cause celebre of any Royal Commission is the rogues’ gallery of wrong doers singled out with adverse findings – the heads on sticks. These were expected in two instalments. Firstly, through the evidence and proceedings of the Commission and lastly, through its Report. In proceedings the suspected villains, the big scalps, the big names were singled out in headlines and sound grabs. Coverage was excoriating at times and visceral, as once powerful politicians and bureaucrats were hammered. But the Report itself, with its secret appendix containing referrals of named individuals, has effectively deferred judgement for now. The report in this sense is frustrating.

Digging in the truth

However long it takes for individual culpability to be assigned in terms of civil or criminal law, if at all, the report is damning in its criticism of individuals. The language pulls no punches. According to the Commissioner, Robodebt was the product of ‘venality, incompetence and cowardice.’[7] The system was crude and cruel, neither fair nor legal.[8]

When all was said and done the Commission did not buy the concocted narrative that Robodebt was the product of bureaucratic mismanagement between two super departments. This was the apologia put by former Heads of the two departments involved, Social Security and Human Services, in the early days of the Commission. It was all just an epic ‘failure of communication’ of sorts. 

The Commission found that Scott Morrison as the responsible Minister should have questioned why a scheme deemed to require legislative change to be lawful in 2015, suddenly became lawful in 2016. Afterall, an Executive Minute supplied to him in early 2015 had described the need for changes to legislation to support the use of annualized PAYG tax data to debt calculation.[9] But this could not rightfully be construed as an assertion of the scheme’s unlawfulness. And senior bureaucrats he claimed had failed to inform him of the 2014 legal advice. 

The idea that records would enable the truth of the matter to be sorted out has so far proved naïve with none discovered that show Ministerial engagement with the issue of lawfulness. The absence of records, suggests a strategy of plausible deniability. But Morrison will have none of it. In comments made since the Report was released, Morrison repeated his claim that the existence of advice pointing to the unlawfulness of the scheme was ‘inexplicably’ withheld from Ministers.[10]   

The Commission also did not buy Scott Morrison’s contention that income averaging as applied by Robodebt was an established practice that had been going on for years or that the system was fair and reasonable to recipients of debt notices. In the case of another Minister for Human Services, Stuart Robert, it did not agree that the convention of collective cabinet responsibility required him to make false statements about the operation of the scheme.[11] Minister Alan Tudge’s ghosting of social security recipients who publicly challenged the fairness the scheme or the accuracy of debts raised was also singled out as an abuse of power.[12]  

For some academic commentators, the over-arching back narrative to all of this was one of a rapacious government, bent on welfare savings that could be directed to other areas of government expenditure. Abuses that came with the system were the product of intentionality and could not be stopped or disowned. For others this was a Government obsessed with neo liberalism and an ideological hatred of welfarism, that would have its pound of flesh from welfare recipients, pour encourage les autres as Voltaire put it. Whatever the case, in 2016 when the scheme ramped up, Morrison was building a narrative of his own about welfare fraud and the Government’s preparedness to be a cop on the beat to protect taxpayers’ money. The shock jocks loved it.

Blood on the floor is going to play out differently with this Royal Commission compared with predecessors. With Robodebt, adverse findings that might once have resulted merely in a wrap of the knuckles, or a mere flesh wound, can now result in something much more serious through referral to the NACC, Australia’s new Federal Anti-Corruption and Crime Commission. While not all misconduct is a pathway to NACC, some conduct certainly is by virtue of the definition of corrupt conduct contained in Section 8 of the Act.[13] At this stage we don’t know the referrals, but actions taken to prolong the operation of an unlawful scheme and prevent its proper scrutiny mean that the digging in the truth is by no means over and that for some, a further process may await further down the track. When these processes gather steam, it may well be that the ‘code of silence’ that appears to have existed between Ministers, their advisors and top bureaucrats about the lawfulness of the Robodebt scheme will come crashing down. 

The questions of who committed wrongdoing, how and why, speak to the role of the Royal commission as an actor-based investigation of the events surrounding Robodebt. If you tuned in to the real time live web feed of hearings or became absorbed by the almost daily media revelations coming from testimony between November 2022 and March 2023, you will have gained this impression of the Commission’s work. 

But the systemic causes of the Robodebt public policy failure is another narrative altogether. The investigation of systems failure does not have the same punter appeal as ‘blood on the floor’ and is a very different task from gathering evidence of people and events behind the unfolding disaster that was Robodebt. But the assurance that reform will be undertaken that will hinder or prevent a future recurrence of Robodebt, is surely one of the most basic things, a sine qua non that we should expect from the recommendations of this Royal Commission. We do not want the bone of Robodebt to be buried and forgotten, leading to the likelihood that it will happen again.

Robodebt and systemic public policy failure

Systemic failures revealed by the Royal Commission are numerous. The Recommendations are grounded in them. While all of these relate specifically to Robodebt, many are also significant as responses to forces that have been shaping government for decades in ways that have eroded good government over time. The irresistible rise of executive government is top of the list. 

If checks and balances on executive power had been operating as they should have, the gestation of Robodebt as an unlawful government program may not have proceeded. And its persistence through the subversion of checks and balances would also have been more difficult to achieve. Two critical checks were the 2017 investigation of the scheme by the Commonwealth Ombudsman and opportunities for review provided by the progression of litigation through the Administrative Appeals Tribunal. The Royal Commission Report describes how both were subverted. 

By the time Robodebt happened, erosion of checks and balances had been taking place over decades. Neoliberal ideas that emphasized the importance of small government, the removal of red tape and rigidities had eroded them. The outsourcing of major reforms and business consultancy to the private sector was also part of this script, functioning to hollow out the expert and experience-based knowledge of the public sector. An audit of employment in the Australian Government released in May, found that the ratio of external labour (FTE) to public servants in the APS workforce was more than one in three and that around 1 in every 4 dollars spent by agencies for departmental purposes was on external labour services.[14] By FTE and portfolio, Social Services was the largest customer for external labour. The most common job families for labour hire were service delivery, ICT and digital solutions and portfolio, program and project management.[15]

Many factors contributed to public policy failure with Robodebt. This was clearly understood by the Royal Commission which explains the breadth of recommendations found in its list. The reform agenda of the Royal Commission is broad encompassing some fifty-seven discrete recommendations. Not all will be recounted here. The Royal Commission aimed for, and mostly achieved an holistic understanding of how Robodebt happened. The systems view is one of an open system where factors such as advocacy, transparency and technology also play a role in good public policy outcomes.

What can be said of the reforms? Some recommended reforms are process oriented and grounded in understanding of the role played by process. For example, the Commission has recommended safeguards aimed at ensuring that New Policy Proposals (NPPs) are lawful and requirements to change legislation are clearly identified.[16] Processes that identify vulnerable welfare recipients and shape how they are engaged are also singled out for reform. Legal foundations to the process of ATO data matching and its governance are also the subject of recommendations. Process failures in recordkeeping are also addressed. The absence of any discussion about the legality of Robodebt found in records points to a conspiracy not to create records. A perennial in the findings of Royal Commissions, the absence of records is deeply worrying and a persistent systemic failure in Government from one Royal Commission to the next. 

In the wake of the Report, much has been made of the erosion of public service culture and its moral compass. The historical role of the APS as a source of ‘fearless and frank advice’ to Ministers looms large in this discussion. As does the issue of bullying by senior bureaucrats and Ministerial office staff whose interests are more clearly aligned with the executive arm of government, than with good government. The undermining of professional codes of ethical conduct, such as in the preparation of legal advice or in the provision IT services, also forms part of the cultural narrative and recommendations for reform. Some reforms in the areas of knowledge and culture are novel. For example, the Royal Commission has recommended that the Commonwealth explore the idea of a ‘knowledge college’ for public servants.

The failure of accountability agencies such as the Ombudsman and Administrative Appeals Tribunal is responded to with a suite of recommendations aimed at improving accountability performance. These are valuable reforms grounded in a very good understanding of how these agencies failed in Robodebt. Likewise there is a recommendation to restore the Administrative Review Council (ARC). In its section on the need for oversight it concludes that because of the ARC’s statutory capacity to inquire into departmental administrative decision-making procedures and the administrative law system more generally, such an agency could have played an important role in exposing the deficiencies of the scheme.[17] Some further expansion in oversight machinery is also contemplated with establishment of a body with the power to monitor and audit automated decision making “with regard to their technical aspects and their impact in respect of fairness , the avoiding of bias and client usability”.[18] 

The Royal Commission Report devotes a Chapter to the nature and role of automated decision making in Robodebt.[19] It makes for disturbing reading if the reader is inclined to a dystopian fear of algorithms, automated decision making and Artificial Intelligence (AI). The savings expected by Morrison and his successors in the welfare spend could only be achieved with automated decision making on an industrial scale without human backup or oversight. The centrality of automated decision making was such that when the system began to generate blow back from wrongly calculated debts, there was no plan to abandon automated decision making, rather a plan was hatched to shift the system to a machine learning methodology based on AI.[20]

Iterations of the Robodebt system were also beset by usability and transparency issues. These belied its poor design and the unfairness of its operation. Recipients of debt notices were provided with no guidance on how the figure arrived at had been calculated. The use of income averaging as an algorithmic method for calculating debts was concealed. All of this took place against a background of technology greenwashing in which the Australian Government had committed to OECD Principles on Artificial Intelligence,[21] the adoption of an Artificial Intelligence Ethics Framework[22] and an Automated Decision-Making Better Practice Guide.[23] 

The Commission itself has not landed on a definitive recommendation on what kind of new or augmented capability is required on automated decision making and AI, rather it has canvassed a range of options including an Administrative Review Council, the Office of the Australian Information Commissioner (OAIC) and an AI Safety Commissioner as once envisaged by the Australian Human Rights Commission.[24] It has also suggested legislative reform. In submissions it was emphasized that a legal framework, aimed at ensuring that algorithms and automated critical decision systems are fit for purpose, lawful, fair, and do not adversely affect human and legal rights, is particularly important where the interests of vulnerable people are concerned.

The Australian Government’s failure to progress an effective regulatory framework for automated decision making and AI was an important root cause of the Robodebt fiasco, albeit not the only one. Australia is not alone in seeking to land on an effective regulatory response to AI and automated decision making. Currently, there is a global fight to contain systems abuse of automated decision making. This fight is shifting from poorly designed algorithms and interfaces, typified by the Online Compliance Intervention (OCI) at the heart of Robodebt, to Artificial Intelligence (AI) and machine learning. When the Royal Commission began its work in late 2022, concern about AI was shaped mainly by deep fakes and facial recognition. In 2023, as the Commission was drafting its report, the arrival of ChatGPT and generative AI, resulted in further alarm, as serious intellects in the developer community declared for the first time, that AI posed an existential threat to humanity. In a rare demonstration of unity from domain knowledge experts, they called for regulation in an Open Letter.[25]  

At the time of Open Letter, the Commission was engaged with the writing of its Report and digesting the submissions it received. There were a few submissions made by academics in the computing and IT space. Some short, some long. Most pointing to the need for urgent action on AI and automated decision-making by the Australian Government. The need for action more broadly outside the context of vulnerable people, can be seen in the private sector, in credit profiling and social media where AI is being used to exploit psychological vulnerability to sell us goods or to influence our politics. It can be seen in war, where AI is working to make the battlefield more lethal, but also to reduce the human presence. 

But discourse on AI is nuanced and case studies in the beneficial application of AI are also numerous. AI is also flavour of the month with Australia’s economists, who see it as a potential source of productivity gains. In health care, AI is working to improve the timeline and reliability of diagnosis, to discover new antibiotics and to treat disease.

Towards a legacy

With the end of this Royal Commission, are we burying Herbert’s bone or delivering something tangible that just might work to reduce the likelihood of further abuse? 

There are many yardsticks by which the legacy of this Royal Commission will be measured. One is the issue of individual culpability where Ministers and senior bureaucrats face a range of allegations including malfeasance in office by pursing an unlawful program, abuse of power and corruption. We will have to wait to see how this plays out. We will also have to wait and see what happens with the reform agenda laid out by the Royal Commission in its fifty-seven recommendations. The Government’s official response will come later in the year. The nature and limitations of that response are likely to be critical in terms of the Royal Commission’s enduring legacy. 

And this is no time to be burying the bone. We should not delude ourselves about the uniqueness of Robodebt nor the potential for further harm from algorithms and AI when used wrongly. The perceived political imperative to reduce the welfare spend has simply shifted focus. It has not disappeared. With the new Labor Government, attention with welfare savings has shifted to the NDIS which is expected to be costing the Government $30 billion annually by 2030. There is clear potential for errors made with algorithms and automated decision-making to repeat wherever temptation exists to use these technologies. 

More broadly, when Commissioner Holmes speaks of the ‘venality, incompetence and cowardice’ that characterized Robodebt, this should prompt a response in thinking people as to what else is out there that speaks to moral failure with these root causes. Well unfortunately, the answer is quite a bit. The persistence of Australian Government support for new coal and gas projects that pose an existential threat to humanity has the same root causes in venality, incompetence and cowardice in our political elite. Another flawed government program, which is persistent, based on wrong assumptions like Robodebt and has more lives than an alley cat, is surely the Australia carbon credits market, presided over by the Clean Energy Regulator. Inherited from the Morrison Government and dismissed as a sham and a fraud by prominent academics and climate scientists, the Albanese Government has stuck with this program.

In a sense the Royal Commission has ended the way most Royal Commissions do – with recommendations aimed at re-invigorating government accountability and integrity. The main threat to a lasting and valuable legacy from the Robodebt Royal Commission is that our political elite is far from enamoured with either. And however much they may bend the knee now or whenever the Government’s response is released, over time they will be working to unwind elements of reform that jar. This has been the history of Royal Commissions that have described substantial reform agendas for government in the past. The capture of politicians by corporations, the money trail of political donations, duopolistic party domination, and the career nature of the political vocation itself, all work to unwind accountability reforms.

The Royal Commission has been a tonic for Australia’s ailing, compromised system of government. How restorative its recommended reforms turn out to be, is a question in which we all have a vital interest as the challenges before government mount. Some of these challenges such as climate change and artificial intelligence are existential in nature. They define crises for society that must be effectively addressed by responsive, energized and honest government, if we are not all to become vulnerable people.

Acknowledgement: This article includes work from a submission to the Robodebt Royal Commission made by the author and fellow information systems professional, Mark Arratoon, a systems engineer and architect with a background in expert systems. The submission is available at: https://robodebt.royalcommission.gov.au/publications/anon-24kg-9s2n-4-and-ano999900010089

ENDNOTES

[1] Candice. (2019). Legal Brew: Demystifying Law and Lawyering. Retrieved from: https://legalbrew.com.au/blog/royal-commissions-need-to-know/

[2] Royal Commission into the Robodebt Scheme. (2023). Report Volume 1 p.xxiv

[3] Burton, T. (2023). Spreadsheet flaw spawned a $4.7 billion Robodebt beast. In The Australian Financial Review, 14 July 2023, p.9

[4] Royal Commission into the Robodebt Scheme (2022). Public Hearing Day 1 Transcript, p.4 Retrieved from https://robodebt.royalcommission.gov.au/system/files/2022-11/transcript-initial-hearing-27-september-2022.pdf p.11

[5] Vide Henriques-Gomes, L. (2020). 'Robodebt-related trauma': the victims still paying for Australia's unlawful welfare crackdown. The Guardian. 21 November 2020. Retrieved from: - https://www.theguardian.com/australia-news/2020/nov/21/robodebt-related…

[6] Royal Commission into the Robodebt Scheme. (2023). Report Volume 2. p.401

[7] Royal Commission into the Robodebt Scheme. (2023). Report Volume 2. p.659

[8] Ibid, v.1 p.xxix

[9] Ibid v.1, p.xxv

[10] Scott Morrison MP. (2023). Statement on Release of Royal Commission Final Report – 7 July 2023. Retrieved from:- https://www.scottmorrisonmp.com.au/news/statement-on-release-of-royal-commission-final-report-7-july-2023/

[11] Royal Commission into the Robodebt Scheme. (2023). Report Volume1. p.302

[12] Ibid., p.179

[13] Commonwealth of Australia. (2022). National Anti-Corruption Commission Act No.88, 2022. Retrieved from:- https://www.legislation.gov.au/Details/C2022A00088/Download

[14] Australian Government. (2023). The Australian Government’s Report on the Audit of Employment. Retrieved from: https://www.finance.gov.au/publications/reviews/australian-governments-report-audit-employment

[15] Ibid., p.9

[16] Royal Commission into the Robodebt Scheme. (2023). Report Volume 1. p.xv

[17] Royal Commission into the Robodebt Scheme. (2023). Report Volume 2. p.484

[18] Ibid., v.1 p.xvi

[19] Vide v.2 Chapter 17 Automated Decision Making, p.469

 [20] Ibid., p.472

[21] Ibid., p.480

 [22] Ibid.,p.479

 [23] Ibid.

[24] Ibid. p485

[25] Centre for AI Safety (2023). Statement on AI Risk: AI experts and public figures express their concern about AI risk. Retrieved from: https://www.safe.ai/statement-on-ai-risk#open-letter

[Opinions expressed are those of the authors and not official policy of Greens WA]