Looking back, casting NLP tasks as QA/MRC (machine reading comprehension) had its moment — see Paradigm Shift in Natural Language Processing from Xipeng Qiu and Xuanjing Huang’s group at Fudan. But most of us, myself included, only ever pointed the trick at the tasks we happened to care about, quietly pleased that so simple a recipe worked so well — well enough to beat the SOTA.
The GPT authors saw further. They noticed that the internet is already littered with task descriptions. So on a parallelism-friendly Transformer, with nothing fancier than next-token prediction, they brute-forced their way across web-scale data — and, step by step, arrived at a genuinely unified NLP model, the first rough outline of a tower of Babel. Worlds apart.
Framing NLU as QA and unifying information extraction as text-to-structure (UIE) were already glimmers of the LLM to come. We just never dared take the next few steps — what was missing wasn’t the technique, it was the imagination, and the nerve.
I felt this one personally: in 2020 my team entered the 8th CCF Big Data & Computing Intelligence Contest (BDCI) with an MRC-based approach, taking a third prize and an “Industry Application Potential” award (writeup, in Chinese). I was standing right at that thin paper barrier — and never pushed through it.