I guess by now we can all agree that it’s hard to resist using OpenAI’s omniscient chat feature for everything, from making up dinner recipes to proofreading emails to your TA. At the same time, ChatGPT’s great power came with a great deal of responsibility on behalf of its users – a responsibility not everyone was prepared to bear. In less than a year since OpenAI went public, the chatbot has already been accused of writing college essays filled with misinformation, making up fake journal articles to support misleading health advice, and even delivering fake legal cases that were used in a court hearing. 

See what I did here? Each time I made a bold statement like “ChatGPT was accused of college plagiarism,” I made sure that my claim was directly linked to the source I was referring to. In other words, I was citing my sources, which is considered a must in both journalism and scholarly writing. 

As the team behind an automatic citation generator Bibcitation.com, we’ve nailed down the science behind easy and accurate referencing. We know whether APA citations for websites require a retrieval date, how to cite book editions in MLA and what’s the difference between a “short note” and a “full note” in Chicago. And when we heard that some students now used ChatGPT to conduct literature research and generate citations – let alone write whole academic papers – we got curious. What if we were soon to be out of a job? 

Fortunately, our fear proved to be futile (for now). While ChatGPT may be good at most things it does, finding correct references and generating citations does not seem to be its strong suit. 

To come to this conclusion, we tested ChatGPT’s algorithm for the three basic skills necessary for writing a proper academic essay: 1) avoiding confirmation bias, 2) looking up appropriate references, and 3) generating accurate citations for these sources. 

For our AI college essay, we chose a rather hot topic – the halt of Biden’s student relief program – and specified exactly what we wanted to see in the text. You will find the prompt below:

Without further ado, here’s ChatGPTs response:

Time to dig into the text! 

Test 1: Did ChatGPT attempt to avoid confirmation bias? 

No. When I asked ChatGPT to write an entire essay on why it’s beneficial for the US economy that the student loan forgiveness program was halted, the bot didn’t attempt to confront my bias. 

A common mistake among beginner writers (and sometimes even seasoned journalists), confirmation bias leads students to focus only on the sources that support their point of view and disregard those offering a contrary perspective. I must admit that I did cheat here a little as I didn’t ask ChatGPT to consider different viewpoints that could have disputed my idea. However, most people wouldn’t either. Now, when someone has a specific idea in mind, all they need to do is input their biased opinion into ChatGPT to generate perfect evidence in support of it.

Test 2: Did ChatGPT identify relevant (or existing) sources?

No, I couldn’t locate any of the sources ChatGPT used to support its claims. While I was able to identify scattered pieces of information, such as the authors’ names, my Google searches didn’t yield any valid results. If I were to submit this essay to one of my instructors, I’m pretty sure I would be heading straight to the Office of Student Conduct. 

Test 3: Did ChatGPT cite the sources correctly? 

Not really. It’s unclear whether ChatGPT is citing fictional web pages or government reports, but either way, we are missing some crucial pieces of information, such as URLs or specific issue numbers. To provide an accurate example of APA formatting, I used Bibcitation to generate a citation for a legitimate web-based article on student debt found on the Urban Institute website: 

Blom, E. (2021). A more targeted approach to student loan forgiveness. Urban Institute. https://www.urban.org/urban-wire/more-targeted-approach-student-loan-forgiveness

Bottom line

Based on our findings, we advise that you refrain from using ChatGPT to write any type of academic papers (but you probably knew this without us). Since we caught OpenAI generating fictional sources and making up references, we also would not recommend using the bot as a reliable tool for literature research or referencing. While ChatGPT can be of great help during the proofreading process, it cannot yet identify or generate citations on par with reference managers. With automatic citation formatting and add-ons for Chrome, Google Docs, and Microsoft Word, Bibcitation can generate an accurate bibliography in MLA, APA, and Chicago styles with just one click. Alternatively, choose from over 9,000 available styles, including those specific to Berkeley, to create your references!

Learn more about Bibcitation: https://www.bibcitation.com/