Digital Welfare State edition 012
DWS Newsletter - edition 12
June 2026
I am writing this in the latest UK heatwave. Any mistakes, typos or omissions can be blamed on the insanity of the weather.
There’s a lot of automated decision making in this month’s newsletter. I’ve just finished writing my book chapter on biased fraud detection systems, so my antennae are highly attuned to it, but it does seem like the outsourcing of consequential decisions to unproven technologies is having a particularly big surge right now.
As always, if you have anything you’d like to share, as well as international news and commentary, or if you’d like to collaborate on a project, please don’t be shy in dropping me a line.
Anna
P.S. if you want to read any previous editions of the newsletter you can find them here, and you can join the generous folk who have made a donation to my costs in putting the newsletter together by giving me a tip. I research and write it without funding or support (or AI).
——————————————————————————————————————————————————-
In which I chat with a chatbot
The DWP has a new AI jobs coach, currently in its beta (testing) phase, so of course I had to have a play with it. There have been some silly articles about it, including one where the journalist pretended to be a parrot. I tried to be a bit fairer in my experiment.
You have to create an account to use it, which might put some people off either because it feels like a faff, or they don’t have the means to (needs an email address and a mobile), or they are unsure about how their details and use of the chatbot might be used by other people (more on that later).
Rather than a parrot, I took on the persona of a mother who had part-time, low paid work in hospitality, and was looking to earn more. She also had a teenage son unsure about what to do after finishing college.
The chatbot gave some helpful information based on my (invented) work history. It asked what I was interested in and about my skills and qualifications, and tailored advice to my circumstances (I said I needed to work around school hours). It offered to search for local jobs based on our discussion. So far, so fine.
It had some suggested shortcut responses at the bottom of the chat which were a bit odd - it kept suggesting ‘I live in Manchester’ which neither I nor my persona do. But it provided useful information on how I could find relevant training and gain qualifications, which is not necessarily something easy to find out by yourself. The job search for roles suitable for the mum persona was decent.
However, when I asked it to search for my fake son, who was interested in media production, the job search was not great - any job with ‘production’ in the title was included, so there were many, many irrelevant ones. That’s not a flaw of the chatbot though, but the job search tool.
If you’ve used any chatbots before you’ll be familiar with the upbeat tone they tend to use. It was nice to have positive feedback and encouraging words, and for people used to more difficult discussions with JobCentre Plus it could be a major improvement. If the chatbot is a signal of a widespread change in approach from JCP, towards a more supportive, encouraging and collaborative mode, I’d be delighted. If it’s not, I worry about the contrast in tone and expectation, and the discrepancy between the positive chatbot and the harsh conditionality and sanctions regime that many people experience.
I do also wonder about the unqualified positivity in terms of how effective it is as a careers adviser. A good careers adviser isn’t just a cheerleader, they will also provide a dose of reality and help their client to build resilience, which requires engaging with the bad as well as the good. I wonder if the chatbot has the capabilities to do this?
Coming back to the issue of privacy, I wondered how my chat might link to the benefit system, and if anyone else would have access to the discussion. One of my big concerns with the tool is that if people suspect that it is being used to monitor their job search and assess their compliance with benefit rules they are not going to trust it. A lack of trust will limit its effectiveness, with people reluctant to share personal information in case it is used against them, or anxious about who might have access to it.
The chatbot won’t answer questions about privacy, and refers you to the Work Hub privacy policy (that’s the government website which hosts the chatbot). There, it states that data provided by users of the service is not used for benefit claims - so this suggests that, for now, interactions with the chatbot will not form part of the conditionality regime. I will watch with interest if that remains the case - I can imagine people within government making a strong case to link them up. If it becomes an instrument of surveillance for conditionality I think it will fail as a tool to help people find work or develop their careers.
The chatbot said I could share the chat if I want to, and that it could be a useful record of my job search activities. It wasn’t clear about whether it would be accepted as proof of meeting benefit requirements though - if this isn’t confirmed I can imagine anyone on benefits who has to look for work to maintain their eligibility would be reluctant to spend much time using it, as they would essentially have to duplicate their job search efforts elsewhere.
So overall, bearing in mind it’s still in beta, and that I am naturally sceptical about using chatbots for public services, I think it’s an ok starting point. It feels like it’s had proper work put into it in terms of the data it is trained on, and the overall tone of responding to personal circumstances in a positive way.
It provided me with some useful information which might have taken me a lot longer to find on my own. I can see it being useful for frontline staff in a council or charity who want to help people navigate their way through jobs and training, and for people who already have some confidence and understanding of what they are asking it about.
I don’t think it can replace a professional careers adviser, and I don’t think it would be able to help someone with complex requirements because, among other reasons, it can’t link the advice it gives to the impact it might have on benefits. I understand why, I don’t expect a chatbot could be trusted to give precise and accurate answers, and it told me the right places to ask for help, but it does mean it’s up to individuals to join up information and advice from multiple, unlinked sources.
We’ll see how it develops as people start to use it (who don’t pretend to be parrots). If you’ve had a go, I’d love to hear your thoughts.
————————————————————————
Things to read
The National Audit Office has reported on the child benefit travel debacle. They identify weaknesses in how risk was managed, and (paraphrasing) how the department cared little for how claimants would be affected. The report suggests that such ‘compliance interventions’ should not be halted, but governed more carefully.
——————————————————————————
More news from Indonesia, with echoes of Kenyan scheme mentioned in my last newsletter. As part of their expansion of digital welfare, applicants for social protection will not only undergo facial recognition, their circumstances will also be analysed in order to ascertain if they are eligible. The system will take things like vehicle ownership and electricity consumption into account. If the Kenyan experience is anything to go by, these automated decisions run the risk of excluding people from essential support.
——————————————————————————
The UK government is introducing facial recognition to judge the age of asylum seekers, despite error rates over 50% for some groups. Internal reporting within the Home Office suggests that the technology tends to over-estimate the age of teenagers, as well as being more likely to say 16 year-olds from West Africa are adults than 16 year-olds from Eastern Europe, and performing worse on female faces than male. A dangerous decision, which (as above) neglects to properly recognise the impact on the people it will affect.
If you want to read more about facial recognition and its use in public services, this is an interesting article on the use of facial recognition in the Indian welfare system.
——————————————————————————
In more news about UK government, technology and jobseeking, a new partnership with LinkedIn will see data from millions of UK users analysed. Information about vacancies, job moves and careers will be used to help government understand more about the labour market. While the idea is laudable, I can’t help thinking LinkedIn is not exactly the source of reliable data, with ‘ghost’ jobs and scammers posting jobs in phishing attempts, and many users perhaps not being entirely honest about their career triumphs.
——————————————————————————
Nonprofit news organisation CalMatters has published details of ‘high risk’ automated systems in use by the California government, including one designed to detect unemployment benefit fraud (yes, fraud detection yet again).
The system, designed by private company Thomson Reuters, combines data from multiple sources, including mobile phone location. It generates risk scores for claimants, generating ‘actionable leads’ and ‘geospatial/network analysis’. These risk scores are meant to predict people more likely to commit fraud, but their efficacy is mixed at best.
I haven’t got space to go into all the reasons why this is a bad idea, but suffice to say, automated fraud prediction isn’t good. It makes significant mistakes, flags people who have done nothing wrong, and in several cases in Europe, displays bias against groups such as women or people from a migrant background. Privacy for those claiming benefits is not a priority, and fighting back against wrongful accusations is normally difficult if not impossible.
——————————————————————————
Speaking of automated fraud detection - this report from Amnesty looks at automated risk profiling in social security, migration and law enforcement. If you want to know more about why automated risk profiling and prediction are bad news, have a read.
——————————————————————————-
And again with the automated fraud detection. Sigh.
——————————————————————————
Finally, more news from DWP. A major project looks set to change how the digital side of Universal Credit works. There isn’t a lot of detail at the moment, as it’s in pre-procurement, but the documentation talks about ‘responsive delivery of policy change, user needs and future service evolution’. There is quite a lot of technical jargon in there (decomposing and microservices anyone?) which I understand is in part about making it quicker and easier to make changes to the system. The spec talks about user-centred outcomes - I would LOVE there to be some user engagement to find out what people who actually use the service would like improved.
——————————————————————————