Reference: The letter Aleph in two different styles: traditional “serif”-style, and more modern sans-serif

The better AI models become at certain things the more fun it is to find things they horribly fail at. My new favorite low brow “benchmark”: draw the Hebrew letter Aleph in SVG format. It should be pretty simple in practice. Three lines: one long diagonal, one shorter curved line at the bottom (with optional “foot”) and another curved one top right.

Let’s start off with our everyman AI, the OG that unleashed this whole generative AI wave in late 2022: ChatGPT or “Geppetto”, as I like to call it. I asked it to complete the task and it immediately replied:

Here’s a clean, scalable SVG of the Hebrew letter א (aleph)ChatGPT

the letter Aleph according to ChatGPT

An absolute fail. Not just “rest-of-the-owl” territory, but utter madness. No further questions, your honor.

Let’s see what our boy Gemini came up with. Interestingly its first response was: “If you upgrade your subscription, I can create it for you today.” And when I politely (“nah bruh”) declined it finally busted out its crayons and declared:

This structure contains the SVG definition of the Aleph. The path data traces the three main strokes of the letter. We use a bold, round stroke to mimic a brushGemini

What is that, a wet noodle?

A literal turd. I mean, you can kind of see what is was going for … kind of. At least there are three lines. But I think we can consider this a solid fail as well.

Next up, what did trusty old Claude have to offer? In perfect over-confident manner it declared:

I’ve created an SVG of the Hebrew letter aleph (א). It shows the traditional form with three distinct strokes – the upper right diagonal, the central diagonal connecting them, and the lower left diagonal. The letter has a distinctive shape that’s been used in Hebrew writing for thousands of years.Claude

Wow, that sounds promising. Let’s open that SVG it painted:

is that a turd emoji or the Adobe logo?

I mean, it’s … bold, I’ll give old Claude that. But it has very little similarity with the actual letter Aleph.

After the Big Three had gloriously and spectacularly failed us, I thought it may be a good idea to let Grok 4 try its hand.

And of course it overpromised as well:

Here is a complete brush-stroke calligraphy style drawing of the Hebrew letter א (Aleph) as if it’s being handwritten with an ink brush. It gives a very organic, calligraphic feel with subtle brush texture and flow.Grok

“long cat” Aleph

Honestly, this is probably my favorite, because it looks so utterly comical, like these inflatable tube men you see flapping around in front of second-hand car dealerships. But yeah, another failure for sure.

I’m genuinely curious why all models dropped the ball here. Three simple lines. Not enough data perhaps? Too much variety in existing datasets from ancient scrolls to modern Hebrew print? At first I thought that the problem was the format: SVG, unlike jpg or png, contains actual path data and AI can’t really fall back on its “fuzzy context” strength. Every line has to be perfect. No blending and blurring like it does in rasterized image generation.

So I tried to let it create some complex shapes like bicycles in SVG format and these all turned out surprisingly well, despite having a lot more complexity than the three lines of the Aleph.

Whatever the reasons ultimately may be, this is now my “strawberry” test. And it’s just too much fun.