This will be brief, as I’m just making a quick stop on the motorway. The saga of the fake psychometric ‘test’, which the Department of Work and Pensions (DWP) has been forcing jobseekers to take under threat of losing their benefits, has just taken a very interesting turn.
Apparently, the US creators of the original test of 150 or so questions, from which the 48 questions for the DWP’s bogus test were taken, have read my blog and the Guardian article that spun off from it, and are going ballistic.
Because the DWP’s ‘nudge unit’ that devised the test as a means of cynically manipulating jobseekers, did not bother to get permission to use the questions in their ‘randomised control trial’ of its effects on hapless jobseekers.
They stole it. And the owners are livid about that, and about how they’ve used the ‘test’ without any of the usual ethical safeguards.
Complaints are being made from the US to the British Psychological Society (BPS) – and others on this side of the Atlantic will be making some too. Including me – and you, if you like.
I wrote the other day about how the test itself is probably illegal in a variety of ways. But that’s not the limit of the government’s criminality. They indulge in a bit of theft, plagiarism and copyright-breaking too.
Please spread the word so the government, especially the DWP, is as shamed and exposed as it deserves to be.
Post by nickd (Mylegal) on May 3, 2013 16:08:46 GMT
More on this in the New Statesman:
Why Downing Street psychologists lied to jobseekers
Nudge Unit may have been trying to use the power of stereotypes and framing to help people get jobs - but that doesn’t mean it was a good idea.
BY JON STONE PUBLISHED 02 MAY 2013
Jobcentres have been foisting a bogus 'personality test' on the unemployed at the behest of Downing Street, bloggers uncovered earlier this week. The tests spat out random 'personal strength'’ to jobseekers who were forced to take part; traits like 'originality' and 'love of learning' – with the feedback apparently having no bearing on the responses people gave.
The ‘My Strengths’ test came from the much-hyped No.10 'Nudge Unit' (officially the Downing Street Behavioural Insight Team), which tries to use discoveries from the behavioural sciences to improve various parts of government. Why would they give the unemployed fake personality test?
One possibility that’s been floated is that they’re pseudoscientific snake oil salesmen with no idea what they’re doing. This is entirely possible. Equally likely, the problems with the seemingly pointless test could also be down to an IT screw-up – it wouldn’t be the first time.
But there is one explanation that I can think of that does make some sense. It’s possible Nudge Unit was trying to use the power of stereotypes and framing to help people get jobs.
There’s quite a lot of evidence to suggest that people 'play up' to stereotypes they have imposed on them. The most famous related experiment is the Stanford Prison Experiment: a random group of subjects are taken and divided into guards and prisoners, then made to staff a pretend prison. After a few days, both groups, picked at random, internalise their roles and the guards are lording it over the prisoners – people play up to the roles you give them. More in the New Statesman here
Post by Patrick Torsney on May 6, 2013 18:52:41 GMT
Oh dear, seems we were right on the permission thing. Here we go today in the Guardian:
Jobseekers' psychometric test 'is a failure'
US institute that devised questionnaire tells 'nudge' unit to stop using it as it failed to be scientifically validated
The 'nudge' unit piloted the psychometric test in Essex despite being refused permission to do so. It has now been rolled out to other areas. Photograph: Danny Lawson/PA
An American psychology organisation has told a UK government agency to stop using a personality test on jobseekers because it is a failure.
The Behavioural Insight team, or "nudge" unit, which was created by David Cameron in 2010 to help people "make better choices", has been accused by the Ohio-based VIA Institute on Character of bad practice after civil servants used VIA's personality tests in pilot experiments in Essex despite being refused permission to do so.
Nudge unit boss David Halpern. Photograph: Felix Clay
The £520,000-a-year Cabinet Office unit run by Dr David Halpern was told by VIA – whose members devised the personality test – to stop using the questionnaire because it had failed its scientific validation.
Official letters sent to jobseekers by the Department for Work and Pensions (DWP) stated that the test was "scientifically shown to find people's strengths".
Kelly Aluise, VIA's communications director, said the institute had previously been approached by a civil servant from the nudge team to use a slimmed-down version of its 120- and 240-question "character strengths" survey.
In correspondence seen by the Guardian, Niemiec said the test was a failure. "They are using the non-validated version … we had tested it a while back and it failed," Niemiec wrote.
In November 2011, the civil servant set up a 48-question version of the test for the Cabinet Office on the website to which jobseekers have been referred to complete the test, which has now been rolled out to other areas of the country.
The Guardian has been informed that complaints against the unit's use of the bogus survey have been lodged with the British Psychological Society and the Health and Care Professions Council, which regulate the practice of registered psychologists.
The DWP confirmed that qualified psychologists – understood to be from the nudge unit – had signed off the project, which was meant to boost confidence and help the unemployed back into work.
Aluise said VIA had asked the Behavioural Insight team to take down its survey and refer jobseekers to their own online version of the questionnaire.
Within hours of the Guardian contacting the Cabinet Office about the issue, the not-for-profit VIA institute, which says it is "dedicated to advancing … evidence-based practices of character strengths", declined to make further comment and said it had resolved its differences with the Behaviourial Insights team.
"Any misunderstandings that may have occurred between VIA and the Behavioural Insights team have been resolved at this point," it said.
The Cabinet Office said the nudge unit – which is being put up for sale – "has a good relationship with VIA, and they are in regular communication".
In response to questions about whether the test was validated, the Cabinet Office backed away from previous written assurances to jobseekers and said the survey was only "based on a scientifically validated questionnaire".
Speaking on BBC Radio 4's Today programme on Friday, Gerry Stoker, professor of politics and governance at the University of Southampton, raised questions about the unit's ethical approval practice.
"What kind of process of ethical intervention have any of these interventions gone through?" he asked.
"When you're deceiving people or potentially … coercing people to be part of something that they don't know they are part of, I think that does raise significant issues."
The saga of the DWP’s fake psychometric test, forced on unwitting jobseekers under threat of ‘sanction’ (immediate cessation of benefits), continues.
After a series of conflicting denials and admissions, and finally an outright admission that jobseekers were ‘directed’ (instructed under threat of sanction) to take the test, I’ve received another remarkable FOI response – this one primarily about the qualifications of the people who ‘coached’ Jobcentre Plus (JCP) advisers how to select the jobseekers who were directed to take the test.
Psychological experiments require stringent ethical and quality controls, carried out by qualified and registered psychologists. The Division of Occupational Psychologists (DOP) of the British Psychological Society (BPS) has been in uproar about the fact that the DWP’s ‘test’ was not properly validated or administered, and the DWP’s senior psychologist is under investigation by the Health and Care Professions Council (HCPC – please forgive all the acronyms, but they’re really unavoidable in this case!) over the maladministration of an unvalidated psychometric test.
This was the context for my latest Freedom of Information request to the DWP. The Department had responded to a previous FOI asking what training was given to the JCP advisers who selected jobseeker ‘lab rats’ for Downing Street’s Behavioural Insights Team’s experiment. The DWP had responded that the training for the test was
part of a facilitated Coaching Session under Building Resilience. Advisers would decide using their knowledge of each individual claimant whether they felt it would be beneficial for the claimant to complete a ‘My Strengths’ test. The sessions were delivered by expertsand the sessions are for coaching and take on the form of a seminar rather than a lecture, hence lack of a script/materials.
Naturally, I wasn’t satisfied by such an obviously vague response, and wanted to know who these ‘experts’ were and what their qualifications were, so I submitted a new FOI to the DWP to that effect. This led to the newest, remarkable response, which you can view in full here, but the key parts of which are shown below along with the questions I asked:
Q: Who were these ‘experts’ [who 'delivered' an informal coaching session to JCP advisers - so informal that there are no notes or written handouts!]?
A: The experts were members of the “Behavioural Insight Team in the Cabinet Office”
Q: What are the qualifications that make them expert?
A: The lead facilitator is a Psychologist in the Cabinet Office.
This is hugely significant, for two main reasons.
First, while the DWP’s chief psychologist is under investigation over this matter, it’s been unclear yet whether the ‘test’ had involved a psychologist at all, or had simply bypassed any qualified involvement. Now we have it on record that a psychologist was involved in this unvalidated, improper, coercive experiment on vulnerable people – but one from the Cabinet Office, not the DWP. Whoever that was, and whoever they report to, needs to be investigated for malpractice too.
Second, and particularly, the BIT’s experts clearly had no qualifications whatever. If they did, the DWP’s response would have used them: ‘the BIT experts were qualified and registered psychologists/psychiatrists/psychotherapists’ or whatever. Instead, they answer a question I didn’t ask, by telling me that the ‘lead facilitator’ (whatever the hell that is) was a psychologist. It’s an evasive answer, clearly designed to cover for the fact that they made a false claim in their previous FOI response. I will be challenging this evasive response!
Q: Please provide a copy of their notes for these sessions – it is implausible to suggest that they all carried out the sessions without any notes or prompts whatever. [The 'sessions' referred to relate to the previous FOI, which asked how the people mandated to complete the 'test' were selected.]
A: As the sessions are for coaching and take on the form of a seminar rather than a lecture, script/materials are not used. I have attached a note which they can refer to which provides an overview of Building Resilience (see separate attachment).
Now we move from the absurd to the ridiculous. These ‘coaching sessions’, supposedly given by ‘experts’ to JCP advisers to prepare them for the rigorous, delicate, vital task of selecting human guinea pigs for a psychological experiment… consist of nothing more than a description of the purported aims of the test. Here it is in its entirety, o you can see whether there’s a single word about how to select candidates or the care that should be taken in doing so, which is what I had asked about:
Building Resilience: Strengths
It widely accepted that increase resilience and positive approaches are important to people in work but we’ve all seen Jobseeker’s Allowance claimants whose morale and self-belief visibly decrease during their time out of work. How do we remedy this?
Often Jobcentre advisers tackle this informally, doing the best they can to improve their claimant’s morale; the best advisers will use resources available on the internet for the claimants to support themselves. The claimant experience in a Jobcentre tends to focus on the weaknesses people have and how we can help them; this is similar to healthcare (think of the language we use like ‘diagnostic’ interview) but it can be more effective to aim for being ‘healthy’ rather than ‘not unhealthy’ – which can lead people to focus on what they are bad at. A light touch approach can stop these issues before they cause harm to people.
How does it work?
We all know confidence in your own abilities is fundamental to finding work (see this article from MEP Seligman as an example) – believing you have something to offer an employer will drive how you sell yourself in your CV, application or interview. In our Jobcentre environment, claimants havenaturally been through a stressful period of either losing their job or having not working for a long time. Strengths tests, like the one below, provide the individual with strengths about their personality they can apply to any scenario. The respondent is not given weaknesses or a score for their strengths – a strength is a strength, not a high or low scoring strength.
Applying it to Jobcentres
As part of the behavioural insights trial, Jobseeker’s are requested to complete a strengths testing exercise at some point in the first 13 weeks of their claim. The exercise is particularly useful if: morale or confidence have visibly fallen the claimant is looking to refresh/update their CV or prepare for an interview The effect of these tests should provide the claimant with a greater belief that they have something to offer an employer, giving them a more positive and optimistic approach to finding work. You can register on the Authentic Happiness website or atwww.viasurvey.co.uk and complete your own strengths test there. Alternatively, you can complete the signature strengths test used in the Behavioural Insights trial here.
Did you find a single word about how to choose candidates, or ethical issues, or the need for care? No, nor did I.
Once again we have from the DWP a series of non-responses designed to mislead and misdirect – yet which give away far more than intended. The current DWP is rather good at persecuting the vulnerable, but rather less good at covering it up.
I’ll be passing this information on to the BPS DOP, so that its members can decide what to do with it. I also plan to make my own complaint to the HCPC, once I have the name of the psychologist responsible for ‘facilitating’ this bizarre, threat-based experiment on struggling jobseekers.