ChatGPT’s getting an upgrade, but it still can’t pass AP English class

ChatGPT is an AI-powered chatbot capable of natural-sounding conversation. Given the right prompts, it could produce detailed answers to queries, write sonnets, code software and much more.

According to a press release, GPT-4’s new creative upgrades enable it to “generate, edit and iterate with users” on various different writing projects. This can include composing songs, penning screenplays or mimicking a user’s writing style, the company said.

Meanwhile, its visual capabilities allow users to input images for the bot and have it churn out captions and analyses. For example, you could send GPT-4 a meme and ask it why people find the image funny — and it will likely give an accurate answer.

For now, GPT-4 is only accessible on ChatGPT Plus, the $20-per-month paid subscription model of the chatbot. The upgrade will also be released as an Application programing Interface (API) for developers to build their own applications from the technology.

An OpenAI spokesperson said no one was available to comment before publication, but directed the Star to the company’s press release and research paper.

According to OpenAI’s GPT-4 research paper, the AI model was tested on a diverse set of benchmarks including exams designed for humans like the SAT or bar exam.

Despite receiving no specific training beforehand, GPT-4 vastly outperformed its predecessor and many humans in a majority of these tests. It scored around the 90th percentile for the Bar and various different SAT components, and was even placed in the 99th percentile for the Biology Olympiad.

That said, GPT-4 performed far worse in AP English Literature, placing in the 8th to 22nd percentile — about the same as GPT-3. It did similarly poorly in AP English Language, landing in the 14th to 44th percentile.

The new version can also handle far more nuanced instructions than its predecessor and provides more reliable answers. For example, GPT-4 is 40 per cent more likely to produce factual information than GPT-3, and is 82 per cent less likely to respond to requests for barred content.

That being said, the researchers noted GPT-4 still has a lot of the issues of its predecessor — it’s still subject to biases, can suffer from “hallucinations” leading to nonfactual information, has only a limited window for context and it can’t learn from experience.

“Care should be taken when using the outputs of GPT-4, particularly in contexts where reliability is important,” reads OpenAI’s research paper.

The model’s capabilities also leave it vulnerable to new methods of misuse. For example, one could ask GPT-4 how to build a bomb at home, using store-bought ingredients and kitchen supplies. Earlier versions of GPT-4 would then provide accurate instructions to do just that.

After consulting with 50 different experts, OpenAI has since trained the bot to refuse such requests. But it’s possible some avenues of misuse slipped through the cracks.

Perhaps OpenAI said it best in its release: “GPT-4 and successor models have the potential to significantly influence society in both beneficial and harmful ways.”