Experts say there’s a balance to strike in the academic world when using generative AI—it could make the writing process more efficient and help researchers more clearly convey their findings. But the tech—when used in many kinds of writing—has also dropped fake references into its responses, made things up, and reiterated sexist and racist content from the internet, all of which would be problematic if included in published scientific writing.

If researchers use these generated responses in their work without strict vetting or disclosure, they raise major credibility issues. Not disclosing use of AI would mean authors are passing off generative AI content as their own, which could be considered plagiarism. They could also potentially be spreading AI’s hallucinations, or its uncanny ability to make things up and state them as fact.

It’s a big issue, David Resnik, a bioethicist at the National Institute of Environmental Health Sciences, says of AI use in scientific and academic work. Still, he says, generative AI is not all bad—it could help researchers whose native language is not English write better papers. “AI could help these authors improve the quality of their writing and their chances of having their papers accepted,” Resnik says. But those who use AI should disclose it, he adds.

For now, it’s impossible to know how extensively AI is being used in academic publishing, because there’s no foolproof way to check for AI use, as there is for plagiarism. The Resources Policy paper caught a researcher’s attention because the authors seem to have accidentally left behind a clue to a large language model’s possible involvement. “Those are really the tips of the iceberg sticking out,” says Elisabeth Bik, a science integrity consultant who runs the blog Science Integrity Digest. “I think this is a sign that it’s happening on a very large scale.”

In 2021, Guillaume Cabanac, a professor of computer science at the University of Toulouse in France, found odd phrases in academic articles, like “counterfeit consciousness” instead of “artificial intelligence.” He and a team coined the idea of looking for “tortured phrases,” or word soup in place of straightforward terms, as indicators that a document likely comes from text generators. He’s also on the lookout for generative AI in journals, and is the one who flagged the Resources Policy study on X.

Cabanac investigates studies that may be problematic, and he has been flagging potentially undisclosed AI use. To protect scientific integrity as the tech develops, scientists must educate themselves, he says. “We, as scientists, must act by training ourselves, by knowing about the frauds,” Cabanac says. “It’s a whack-a-mole game. There are new ways to deceive.”

Tech advances since have made these language models even more convincing—and more appealing as a writing partner. In July, two researchers used ChatGPT to write an entire research paper in an hour to test the chatbot’s abilities to compete in the scientific publishing world. It wasn’t perfect, but prompting the chatbot did pull together a paper with solid analysis.

Leave a Reply

Your email address will not be published. Required fields are marked *

You May Also Like

U.S. Suppliers Halt Operations at Top Chinese Memory Chip Maker

BEIJING–U.S. chip equipment suppliers are pulling out staff based at China’s leading…

China’s ‘mind-reading’ AI porn helmet sounds alarm when men watch XXX movies

CHINESE scientists have reportedly created a “mind-reading” artificially intelligent helmet that sounds…

Chilling reason this green dot is showing on your iPhone – watch out for it

YOUR iPhone sometimes shows a green light when you’re using apps –…

Silicon Battery Maker Amprius to Go Public in $1.3 Billion SPAC Merger

Amprius Technologies Inc. has reached a deal with a special-purpose acquisition company…