Digital Welfare State edition 011
DWS Newsletter - edition 11
May 2026
After a relatively quiet April, there is LOADS to read this month: news from Indonesia and the UK, opinion from Pakistan, research from South Africa and brilliant investigative journalism in Kenya. I recommend getting a drink (a very cold one if you’re enduring the heat that’s overwhelming many of us) and setting aside half an hour to enjoy it.
This edition opens with a guest post from Gabriel Geiger, an investigative reporter from Lighthouse Reports. Gabriel has been part of multiple investigative teams looking into automated welfare systems; if you’ve read about cases of flawed systems in Rotterdam, Sweden or Amsterdam, chances are you’ve seen his work. This post looks at an AI system, the Social Health Authority, used in Kenya to assess how much people should pay for healthcare
As always, if you have anything you’d like to share, as well as international news and commentary, or if you’d like to collaborate on a project, please don’t be shy in dropping me a line.
Anna
P.S. if you want to read any previous editions of the newsletter you can find them here, and you can join the generous folk who have made a donation to my costs in putting the newsletter together by giving me a tip. I research and write it without funding or support (or AI).
——————————————————————————
Kenya’s AI health mistake - guest post from Gabriel Geiger
Away from the hype and fear marketing of Big Tech, AI is doing real and current harm to vulnerable populations. These harms are compounded by being under-reported and often invisible.
A population-level AI experiment to predict how much Kenyans could afford to pay for access to healthcare is a life or death example. Lighthouse Reports spent 18 months working with Africa Uncensored to gain access to and take apart this system.
What we found is a warning to governments, civil society and tech policy watchdogs everywhere. Under the guise of “neutral tech” the system was knowingly designed to maximise revenue by shifting the burden of its mistakes onto the poor.
KEY AUDIT findings:
The poorest pay more than they could or should. The system disastrously overestimates the earnings of the poorest and underestimates the earnings of the richest.
The government knew. We obtained a consultants’ report that warned the system was “inequitable”, based on flawed data and “highly likely” to miscalculate.
Low-income households would be hardest hit. Insiders told us how at every turn their warnings about the tech were ignored as the changes were “bulldozed through”.
The SHA system uses proxies like access to a toilet or running water, ownership of a radio or phone; building materials used in homes to estimate income and calculate health premiums. In a country where some 80 percent of people are in the informal economy an inaccurate answer affects the lives of millions of people.
Grace, a community health worker charged with signing up people for SHA, who couldn’t afford her own AI predicted premiums, told us “people are dying, people are suffering.”
Read and share the interactive long read telling the full story. See the Error By Design documentary, and check out this short video that tells the backstory of how this extraordinary investigation came about from the two reporters who led the work. See the write up from our partners at the Guardian.
——————————————————————————-
Things to read
News from Indonesia that a small-scale pilot of a digital social protection system is being scaled up to reach over 36 million people. The system, designed to make it easier for people to apply for and access benefits that they are eligible for, will expand to 42 cities and districts. Its basis seems to be the integration of data and identity records to overcome barriers to access. It will be interesting to keep an eye on this - does it indeed expand access and improve the targeting of support, or does it lead to the same kind of exclusion that we have seen in countries such as India.
Interestingly, the story linked immediately underneath is about people in receipt of social assistance being penalised for spending some of their money on gambling. ‘Data matching’ was used to link people who received aid with online gambling transactions, with those identified removed from the official list of families who qualify for aid. I’d love to know more about the sources of data, what provisions there are in place for privacy, and if the government has assessed the accuracy of their data matching. If anyone knows more, please get in touch.
——————————————————————————-
Another intriguing story - DWP has entered into a contract with a company to use AI for ‘sentiment analysis’. It’s not obvious what they are using it for; previously it was used to analyse posts on social media to give some insight into what people through of the department and its services. DWP have used (are using?) different software to analyse social media posts with a view to identifying fraudulent claims. The Cosain software from Capita can analyse thousands of posts a day, and create alerts which presumably notify officials about language or content which may signal fraudulent activity. This raises loads of questions - not least, what are they looking for when they analyse posts, and who decides which claimants they are going to look into? Is the new sentiment analysis software part of anti-fraud measures, and how do all of their measures knit together?
——————————————————————————-
The UK government has released a new AI chatbot which is designed to help people navigate government websites and find accurate information quickly. Their description suggests it will help to free up staff time for more complex cases and enquiries. If you are in a public-facing role and have heard from anyone who has used it, for example to search for information on applying for benefits, I’d love to hear about the experience.
——————————————————————————-
This is a clear and useful article on the problems of using algorithms to try to solve complex issues like identifying welfare fraud. It points out the political and cultural underpinnings of using so-called neutral tools like algorithms, and why challenges like welfare fraud detection are wicked problems, in which there is no formulaic or compromise-free solution. I’ve been exploring these issues for a book chapter I’m writing at the moment - watch this space for more details later in the year.
——————————————————————————-
A really thought-provoking article on Pakistan’s Benazir Income Support Programme (BISP) which since its launch in 2008 has become a digitised programme, transferring cash directly to women. The article argues that BISP has played a crucial role in recognising the unpaid labour carried out by women, and due to its digital nature has also provided a huge stimulus for women’s digital and financial inclusion. Prior to acquiring a digital identity needed to receive BISP, many women, particularly rural women, had no independent contact with government agencies, or influence over money in a household was spent.
Of course, becoming digitally identifiable is not without potential drawbacks, and I’m sure the digitisation of BISP has not been entirely straightforward, but the article makes a good case for the inclusion benefits of digital welfare programmes, as well as the impact on poverty of direct cash transfer schemes. If you’re familiar with the digital development of BISP and how it manifests in the lives of Pakistani women let me know.
——————————————————————————-
I love this research report from the Institute for Development Studies, looking at how accountability can be built between citizens and digital welfare systems. It draws on experiences in South Africa, where digitisation was viewed as inverting traditional accountability roles, making citizens accountable to the state rather than the other way round. It describes how citizens in South Africa were involved in developing accountability mechanisms which incorporate citizen voice and provides some useful lessons for civil society actors in any country looking to enhance citizen voice and involvement.
——————————————————————————-
If you want to get angry, read this post by Luke Farrell about the insane amounts of money spent by US government departments accessing and verifying data about citizens as part of many of its welfare programmes such as Medicaid. Millions goes to private contractors, with the price of the services they provide rising rapidly, and departments paying wildly different amounts for the same data. He argues that this level of reliance and vendor lock-in is not inevitable, but in the US right now the political direction of travel seems to be overwhelming.
——————————————————————————-
This is an amazing resource for anyone in or interested in Latin American and the Caribbean - a database of AI systems being used in the public sector. There are chatbots, vehicle control systems, facial recognition tools, document handling systems and lots more. It’s a bit overwhelming initially, but looks really searchable. Have a look around.
——————————————————————————-
Transparency fail at the UK Home Office. People claiming asylum in the UK are not told that AI is being used when processing their applications. AI is being used to ‘extract and analyse information’ from interview transcripts and help officials to query policy documents. The department says no decisions are made solely using AI, and that it does not release information to asylum applicants on the tools it uses. I’d be keen to see how they are ensuring accuracy in the extraction of information, given what we know about AI ‘hallucinations’ and a general lack of accuracy from many commonly used AI tools.
——————————————————————————-