It started Erev Pesach with a bottle of olive oil.

“What are we having for dessert?” I muttered to myself as I looked at my laptop. My last-days-of-Pesach menu stared back at me, every meal planned with dips, soup, main, and sides. My husband and I had agreed to host last-minute guests for meals, and I wanted to serve something more elevated than the cut fruit I had originally planned.

Something. But what? I left my laptop and ducked into the kitchen, looking for ingredient inspiration. I found cocoa powder, almond flour, and sugar — not a bad start.

Maybe brownies, I thought, and turned to hunt for some vegetable oil so I could start baking. Only there was no vegetable oil. I had run out over Chol Hamoed. All I had left was a nearly empty bottle of extra-virgin olive oil. You know, the kind for salad dressings and savory sautés and definitely not brownies?

I sighed and went back to my desk, resigned to the fact that we would just have fruit for dessert. I didn’t have time to either shop for new ingredients or trawl through the Internet for a Pesach dessert recipe involving olive oil. I decided to take my mind off the disappointment by knocking a few things off my to-do list for work and opened my project-planning app.

My first task was to draft an email informing a group of students about an upcoming exam. I opened a new tab and navigated to ChatGPT, the artificial intelligence chatbot that I had been using to write messages for the past several weeks in order to boost my productivity. I prompted the bot with my request, and immediately, a professionally worded email draft started to appear. As I watched the text grow on the screen, an idea began to grow in my mind.

I started a new chat with the prompt: “Generate a dessert recipe based on almond flour that includes olive oil. The recipe must be gluten-free, dairy-free, and suitable for Passover.” Within seconds, I had a recipe. Within minutes, I had dessert in the oven. And within a few hours, I was serving a delicious Pesachdig orange-olive oil cake to my guests, to rave reviews.

What Is ChatGPT, and How Does It Generate Recipes?

Out of curiosity, I asked ChatGPT how it generated the cake recipe. I received this response:

As a language model, I was able to generate that recipe by utilizing my natural language processing capabilities and knowledge of cooking.

First, I searched for a recipe that included almond flour and olive oil, and then I narrowed my search to recipes that are dairy-free, gluten-free, and suitable for Passover. I then used my knowledge of baking and cooking techniques to create a recipe that included the appropriate measurements, ingredients, and baking instructions.

While my recipe may not have been tested in a kitchen, it was generated using ingredients and techniques that are commonly used in almond flour baking recipes. I hope you find the recipe useful and enjoy making it!

The Goal

After this initial success, I was curious to learn whether ChatGPT could consistently produce reliable and delicious recipes — or whether I had experienced beginner’s luck. My goal was to use ChatGPT to generate a full Shabbos dinner menu, including challah, two different dips, a soup, a salad, one main dish, two sides, and a dessert.

Initial Impressions and Challenges

To start, I hopped onto ChatGPT and prompted it to create a “Shabbos dinner menu that adhered to the laws of kashrus” and contained the courses I had decided upon. The bot spat out a stereotypical mix of dishes — gefilte fish and chopped liver, matzah ball soup, brisket, and babka. Ever the considerate chat bot, it even added a note at the end of the menu that I should make sure to consult my local “rabbi or other knowledgeable authority regarding the specific dietary requirements for [my] community.” It seems that while ChatGPT has a narrow understanding of what can be served at a Shabbos seudah, it does know about asking an LOR.

I then requested a menu that was more unique and flavorful, and was immediately rewarded with a menu filled with warm Middle Eastern spices that included baba ghanoush, Moroccan lentil soup, and orange semolina cake. There was just one issue — the main was chicken shawarma sandwiches, which I’d happily serve for a weekday meal, but not as a main for Shabbos.

I decided to keep experimenting with different prompts, which taught me a few things about how ChatGPT currently functions when generating recipes. First, I found that if I made multiple menu requests within the same chat, the bot would “forget” about the kosher requirements and give me recipes involving dairy and bacon. I had to “remind” the bot multiple times to generate recipes that were dairy-free and suitable for a kosher meat meal.

I also found that the way I worded my prompts greatly impacted the results. Writing “Shabbos” or “Friday night dinner” in the prompt would invariably return menus with recognizably Jewish food, such as kugels for side dishes. Including “dairy-free” or “pareve” in the prompt would give me results from the vegan world, including vegan apple crisp, chocolate-date truffles, and cookie recipes without dairy and eggs. Asking for full menus usually resulted in simpler recipes, like simple roasted chicken, while asking for individual recipes with specific descriptors (like unique, avant-garde, easy) generated more exciting results.