Deep learning is not learning. The clue is in the need to put *deep* in front.
A quantified estimate of the difference between learning and deep learning.
A computer scientist who I highly respect said `Computer science is not a science. The clue is in the need to say science. No one says biology science, or physics science. It is just biology. Physics. If it is a science you don’t need to add the word. Same to neuroscience, it is not science, or why do you say the word in front?’.
It is already the time to clarify matters for deep learning.
I never did any kind of deep learning and I hope I never will. It sounds very painful. Who has ever said “I have done some very deep learning last night.” Who has ever said: “My child is very smart because their learning is of the *deep* kind.” We never needed to say *deep* together with *learning* until it was done by a machine, until the time when the learner was not alive.
More important, in my opinion, is the debate: Is deep learning, learning? And here I feel very comfortable articulating an opinion, in my authority as a living system.
There are two technologies in question: Large Language Models (LLMs) and living stuff. We seek an objective comparison between two technologies and their associated hardware. What do we know for a fact?
TECHNOLOGY ONE (TECH1)
Learning by non-living systems (LLMs and associated fauna).
Hardware: Sequencing models (stuff on a computer somewhere).
Technology knowledge: Let us be conservative and say that generative models are based on the whole history of human knowledge. Starting with the Egyptians. So LLM technology is based on 104 years.
Technology age: 104 years.
Researchers of technology: human researchers.
Drive: occupational job.
TECHNOLOGY TWO (TECH2)
Learning by living systems (including learning by humans)
Hardware: Bunch of particles that make up a living system, human is one example.
Technology knowledge: This technology started 4 billion years ago. Driven by living organisms that are learning about the world around them at the cost of not surviving.
Technology age: 109 years.
Researchers of technology: any collection of atoms that are trying to exist.
Which one can learn better? Can technology one replicate what technology two does?
There are five orders of magnitude difference between the ages of TECH1 and of TECH2. Is it possible that TECH1 does the same as TECH2? It is not impossible but my money is on TECH2. We are talking about technologies that are 104 years versus 109 years in the making.
Living systems have been developing their learning for 4 billion years. Human researchers have been developing their understanding of learning for 10 thousand years. 5 orders of magnitude difference in the knowledge development? Without need for any further details my money is on TECH2.
Then you look at the motivation of the researchers of TECH1 and TECH2. There have never been any scientists doing computer science at gunpoint, fighting for their life (thank goodness).
Living systems have done nothing other than this. How much harder will you explore the properties of reality, the properties of the natural world around you, when you know that your bare existence is at stake.
Just because a living system, like a human or a worm, learns, it does not mean it knows how to make something else learn. If the human itself does not know how it learnt, how can it build a recipe, develop code, for how to make something else learn?
This is the same quest of pedagogy, and is best encapsulated in the quote by the pedagogue William Hull: “If we taught children to speak, they would never learn.”
We are humans v.4billion.
Deep learning is not learning. The clue is in the *deep*.