A couple of weeks ago we made a decision we have never had to take before. We removed two applications from a shortlist for an internship because we realised they had been created using AI. 

As I write, it is the busiest point of our application calendar here at The Brokerage with our placements team shortlisting hundreds of applications for our summer internships. Hundreds of young people, sixth-formers and undergraduates, are applying for paid internships at our partner companies. The vast majority are writing applications without AI, but this is the first year where we have noticed applicants using it.

How should we approach this? How can we tell if an application is AI generated? Does it even matter?

Some larger organisations will be in a position to employ sophisticated technological solutions to weed out ‘cheating’ applications. Right now, for The Brokerage and our parnters – many of whom usually have small recruitment teams – that isn’t an option. So we’re relying on spotting AI applications during the manual stage of the shortlisting process. What have we learned?

Knowledge based questions are where young people are most likely to use AI

Application forms for our internships include, alongside other information, two questions that require longer answers. The questions are set in conjunction with the employer and are used to understand the applicants interest in and suitability for the role. 

Questions that ask about knowledge – ‘What are the interesting new developments in our industry?’ – are the ones where candidates are most tempted to use AI. Maybe if someone feels they will have to do some research anyway, they might as well put the question into an AI tool instead of google. 

The problem with this is that we have ended up with lots of applications where almost identical answers have been provided to the same question. It is relatively easy to spot, when shortlisting a large number of candidates. 

The solution to this is probably simple – if you need to ask about knowledge, do it in the interview. Use the application form to find out about the individual applicant. Ask questions that only they could answer. ‘What skills have you gained from your previous experiences, and how do they make you suitable for this role?’ is a question AI is much less likely to be able to help with. 

AI has a style that is pretty easy to spot – at the moment 

At The Brokerage we are very used to the written style of our candidates. We work with young people aged 16 to 25 and these internships are usually aimed at the younger end of that age group. People in this age group have usually written fewer job applications than older people and have engaged with less ‘business English’, but they have written essays and university personal statements. That means young people have a style that is not the same as business English. The grammatically perfect, but slightly cliched English style of AI does not read like an 18 year old writing an application. 

So the solution here is also simple: think about the style you’re reading. Is it too perfect? If so, it might be worth checking with one of the various online tools that can predict if a piece of writing is AI generated.

The world has changed – and we need to think about how job applications need to change to keep up

In the same way that calculators mean anyone can find out the answers to a mathematical equation, AI means anyone can create a coherent passage of text. That is a real change and recruiters need to know how to deal with it. 

But is it a revolutionary change? Anyone who has ever interviewed people for a job knows that there is a difference between the skill of being able to answer coherently and confidently, and actually being able to do the job. A well designed interview process deals with this by trying to test the skills that matter, either through carefully selected questions, additional tests and tasks, or both. 

Written tasks now need the same thought put into them. Do you need the employee to be able to write independently, or are you happy for them to use a ‘calculator’ for their written tasks? If it is an important skill for the job you’ll need to test that skill specifically. That could be via carefully chosen questions at the start of the process, by a written task during the assessment stage, or a combination of the two. 

IS AI overhyped?

It is too early to say how big an impact AI is likely to have, that’s the sort of judgement that can only be made with the benefit of hindsight. In the short term, job applicants are using it. Employers will be caught out if they don’t stop and think carefully about the questions they are asking and how they are asking them.