It is not possible to define the “essence” of a problem.

You can't productively reason about the “complexity” of a problem in isolation. There is only complexity of solutions. Sometimes the only solutions we can imagine are complex, and so we call the problem complex. But then someone comes up with a theorem, or computers just get way faster, and suddenly the solutions are simple.

Has the complexity of the problem changed? No, simply the tools with which we can solve it.

The problem never had any “complexity” to begin with, essential or otherwise.

Fred Brooks wrote about “essential” and “accidental” complexity in his famous essay No Silver Bullet, but Dan Luu argues pretty persuasively that his breakdown of “essential” and “accidental” is useless:

Revisiting Brooks on computer performance, even though I deal with complexity due to the limitations of hardware performance in 2020 and would love to have faster computers today, Brooks wrote off faster hardware as pretty much not improving developer productivity in 1986:

What gains are to be expected for the software art from the certain and rapid increase in the power and memory capacity of the individual workstation? Well, how many MIPS can one use fruitfully? The composition and editing of programs and documents is fully supported by today's speeds. Compiling could stand a boost, but a factor of 10 in machine speed would surely . . .

But this is wrong on at least two levels. First, if I had access to faster computers, a huge amount of my accidental complexity would go away (if computers were powerful enough, I wouldn't need complex tools like Presto; I could just run a query on my local computer). We have much faster computers now, but it's still true that having faster computers would make many involved engineering tasks trivial. As James Hague notes, in the mid-80s, writing a spellchecker was a serious engineering problem due to performance constraints.

Luu argues that faster computers reduce accidental complexity. I argue they simply change the problem.

Luu specifically calls Brooks out for the phrase “how many MIPS can one use fruitfully?” Time has obviously proven Brooks wrong here. We would not have video editors, chess computers, translator apps, or modern video games without dramatic advances in computational power. (Although perhaps Brooks would not consider video games to be “fruitful”.)

Faster computers changed the problem. Brooks's idea of “complexity” was tied to his solutions, but his solutions are irrelevant now.

I don't think the idea of essential and accidental complexity is completely useless. It's very easy to identify accidental complexity in many places. We can easily judge if part of a solution is “more essential” or “less essential”. And in the end, this is all we need.

I personally prefer the terms I learned from Demetri Spanos on Handmade Network: direct / indirect. Using the tools you have, are you solving your problem directly? Or are you solving it indirectly? Are you solving the problem in front of you, or are you trying to solve other problems at the same time?

We don't need to know the exact “essence” of a problem in order to cut accidental complexity from our solutions. The more essential, the better, but I don’t need an exact number. I’m happy just moving in the right direction.