Will the humans who created artificial intelligence (AI) soon be relegated to some dusty corner?

“What do you think about this?” my chavrusa asked me one morning.

He showed me a short text explaining the advantages of being an avreich:

Being an avreich is a very meaningful and noble way to serve your community. As an avreich, you are able to provide an important service to your community by teaching Torah and providing spiritual guidance. You are also able to share your knowledge and wisdom with others and help shape the future of the Jewish people. Additionally, being an avreich is a great way to deepen your own understanding of Jewish texts and to enrich your own spiritual life.

“So what do you say? Is it worded well?”

A bit kitschy and superficial, I thought to myself, but not bad, overall.

My chavrusa told me it had been generated by artificial intelligence (AI). He directed me to a new app, ChatGPT, which had set a record of a million downloads in a week. Developed by OpenAI, a San Francisco nonprofit cofounded by Elon Musk, Peter Thiel, Sam Altman, and others, ChatGPT is able to mimic human communication on a very high level.

I checked out this latest high-tech wonder. If you haven’t heard of it by now, you certainly will. It can do almost anything: write a syllabus for an education course, develop a detailed plan for establishing a start-up, compose songs, or even write lines of code by itself, saving you the need to hire a human programmer. It can come up with a list of attractions to see on a trip, or just give you a recipe for a good soup on a wintry day.

My curiosity mounted. I decided to try it out myself, and test its skills on that inimitable genre, frum music.

“Can you create an MBD-style Jewish song about Jerusalem?” I asked the interface.

I was late to the game, as users had been testing the software on music lyrics from day one. An (uncomfirmed) version of an Avraham Fried-themed song about Mashiach had made the rounds, and the resulting chorus was somewhere between Fried and a parody of the Red Army Choir:

“He’s the one we’ve been waiting for,

The savior of the Jewish nation.

With him by our side,

We’ll find true liberation.”

But by the time the MBD request was made, pushback from artists who complained about copyright infringement had left its mark, and ChatGPT’s songwriting functions had apparently been disabled.

“I’m sorry, I am not able to create a song in any specific style, including that of MBD,” the software responded. “However, I can provide you with some lyrics in Yiddish about Jerusalem that you could potentially use for a song:

“Jerusalem, di goldene medina,

Di shtot fun unzere vayse vaytn,

Di shtot fun unzere nakhes un freyd,

Di shtot fun unzere tratsn un raytn.”

Actually, I was touched by the app’s honesty. It had burst onto the scene with great fanfare just a few weeks ago, and now already its wings are being clipped. Who says an algorithm can’t feel? And besides, who decided we should prevent it from actualizing its unique abilities?

Giving up on ChatGPT’s artistic abilities, I switched to another subject — albeit a closely related one. We began to chat about the history of Jerusalem. And here I first began to see the app’s significant faults. The very articulate and organized text it presented me contained a claim that I knew to be groundless — if not outright conspiratorial.

I ran the results by a researcher, who confirmed that the app occasionally cranks out nuggets of nonsense. He told me about an entire scientific article that the program presented, citing headings and abstracts that never existed. It does this because, as impressive as it seems, it doesn’t “understand” what it is saying. It often tries to guess at what is most likely the next word — and if the result squares with the algorithm, then the app considers it reasonable, even if it is completely false.

Okay, so the unbelievably polite program that I was introduced to turned out to be a pathological liar, and it lies in this authoritative, irritating, know-it-all way. But that is not really the disturbing issue. When I tried to ask the program where it got its fabricated information, it could not direct me to the sources. And putting aside the hype about AI’s current capabilities and the very real fears of its effect on jobs, that aspect is one of the biggest problems with the new technology.