We’ve been running an experiment behind-the-scenes. Who is better, a trained CIPS examiner or Microsoft Copilot? Here, we unveil the advantages (and disadvantages) of using A.I. to mark your mock exams and get you ready for that all-important CIPS exam day!
How does it work?
For some time now, artificial intelligence has been able to provide generative data to help inform and coach you through education. It’s no different for CIPS, and there are many legitimate uses of A.I. to help you through your studies.
When preparing mock exam essays as part of your exam preparation, it would obviously be counter-productive (and unethical) to ask A.I. to write your essay answer for you. There is very little to be gained from that.
However, what about marking your mock answer once you’ve written it?
Here at Cordie we ask all CIPS learners to complete mock exams for their Constructed Response (CR) exams. Our tutors mark the mock papers, provide an indicative grade and mark, and provide constructive feedback on what’s good about the answer and what needs to be done to improve it further.
But A.I. can do all of this - and faster!
However, that doesn’t mean it is any better - and this is what we have been experimenting with in recent months.
What can A.I. do for student exams?
It is relatively simple. You upload your exam answer into your favoured A.I. application (…we used MS Copilot, but obviously there are others) and you ask it to make an assessment of your answer, provide you with a mark and show you where it could be improved upon.
Naturally you set the parameters by explaining which CIPS module and which syllabus it is from.
You can also ask A.I. to mark more critically if you feel it is softsoaping its response back to you.
Your A.I. application will come back with a breakdown of marks for each question and provide a justification for its marks, set in context of the CIPS syllabus and the exam question command words, and then go on to tell you where you think your answers could be improved upon.
Obviously this is extremely helpful feedback information that is able to come back to you almost in an instant.
But how good is A.I. at this?
In our tests, we approached a trained and experienced CIPS assessor who has been marking CIPS papers (with standardisation and moderation) since 2004. We asked them to mark a series of student mock papers and provide an indicative mark. We then put the same mock answers through Copilot and compared the results:
In general, MS Copilot is a lot more generous than the expected standards from CIPS. Our experiment suggested that Copilot tends to over-mark by approximately +12% (meaning that if the CIPS examiner awarded your paper a 50% mark, Copilot would have typically awarded 56% or more). This has the danger of lulling you into a false sense of security in the lead up to your exam.
The feedback from Copilot was written positively and supportively. This is particularly important (and good), because learners are relying on encouragement to build their confidence and develop their skills. This is a positive aspect of Copilot.
A lot of the feedback we received from Copilot was obvious and did not really add much value. Comments like: “the exam question asked you for five explanations, but you only provided three” are fairly self-explanatory. The feedback for improvement is slightly patronising to suggest: “you should have selected five [in order to get more marks].”
Copilot draws on a lot of general knowledge and useful peripheral information. The feedback helps to contextualise and give general recommendations for improvement and/or additional theoretical content. However, there is insufficient application to the specific content of the CIPS syllabus and so the technical feedback from Copilot lacks specific depth and quality.
So, what do you recommend?
Based on our experiment, we believe there is merit in learners (and in particular self-study students) using A.I. as an initial feedback mechanism on mock answers.
However, please be careful not to over-rely on the feedback it provides.
Our research suggests A.I. is not sufficiently critical yet and it tends to be over-generous in its marking.
You are still better off getting the critical view of an experienced CIPS tutor / examiner. They know the detail of the syllabus and they understand what CIPS is looking for from its learners. A.I. is a helpful start, but it is not sufficiently well developed yet to replace the insight and expert guidance of a CIPS tutor (yet!).
Please also be careful with the information you are inputting to your A.I. application. You must refrain from entering personal data and/or commercial information, as this is likely to be a data security breach!
We hope you have found this study blog article helpful. Please get in touch with your own thoughts or experiences about using A.I. to help study for your CIPS exams.
If you would like any further information on the services we provide and/or the study workshops available, please get in touch via the contact form at the bottom of this page.
We look forward to hearing from you!