A constriction on GPUs is literally the best thing to ever happen to Chinese ML dev.
It made them thrifty, it made them focus, it forced them to go open weights, it made them build proper ASICs, research new techniques, pay engineers to implement them, and now their models are supremely efficient, dirt cheap, running Nvidia free on Huawei NPUs, and close to better tools than the US models.
Meanwhile, US models are all (except maybe Google) enshittifying and getting benchmaxxed. Engineers are wasting man hours hopelessly trying to scale training, which does not scale like people think, and are literally giving GPUs busywork to meet utilization quotas. They’re trying to scale data and parameter count, without improving architecture or data quality or even basic problems like random token sampling, and it’s not working anymore.
At the same time, the big US AI houses have squashed nearly every bit of “garage innovation” I’ve seen. Cool teams, hero devs with proven work on a budget, they all just disappear into the maw of Microsoft or whomever like it’s a black hole, their work never integrated into anything.
US AI is GOING to collapse because we gave all the money to tech bros so they can poison the well. The ML research community has been screaming this since like 2022. And apparently before, as Aaron Swartz allegedly identified Altman as a sociopath right before he died by suicide.
Sorry to rant.
Not that China doesn’t have significant dev issues, to be clear.
Europe, too.
But this is a sensitive point for me. Hobbyist machine learning has been a passion of mine for a decade, and it makes me sick to hear people quote Altman, like throwing GPUs at tech bros going to fix this. That. Is. A. LIE.
I don’t have a solution either. In the AI space, I do not even see a path back to moonshot-style cooperative innovation like the US has repeatedly pulled off before.
A constriction on GPUs is literally the best thing to ever happen to Chinese ML dev.
It made them thrifty, it made them focus, it forced them to go open weights, it made them build proper ASICs, research new techniques, pay engineers to implement them, and now their models are supremely efficient, dirt cheap, running Nvidia free on Huawei NPUs, and close to better tools than the US models.
Meanwhile, US models are all (except maybe Google) enshittifying and getting benchmaxxed. Engineers are wasting man hours hopelessly trying to scale training, which does not scale like people think, and are literally giving GPUs busywork to meet utilization quotas. They’re trying to scale data and parameter count, without improving architecture or data quality or even basic problems like random token sampling, and it’s not working anymore.
At the same time, the big US AI houses have squashed nearly every bit of “garage innovation” I’ve seen. Cool teams, hero devs with proven work on a budget, they all just disappear into the maw of Microsoft or whomever like it’s a black hole, their work never integrated into anything.
US AI is GOING to collapse because we gave all the money to tech bros so they can poison the well. The ML research community has been screaming this since like 2022. And apparently before, as Aaron Swartz allegedly identified Altman as a sociopath right before he died by suicide.
Sorry to rant.
Not that China doesn’t have significant dev issues, to be clear.
Europe, too.
But this is a sensitive point for me. Hobbyist machine learning has been a passion of mine for a decade, and it makes me sick to hear people quote Altman, like throwing GPUs at tech bros going to fix this. That. Is. A. LIE.
I don’t have a solution either. In the AI space, I do not even see a path back to moonshot-style cooperative innovation like the US has repeatedly pulled off before.