

I agree that you can’t know if the AI has been deliberately trained to act nefarious given the right circumstances. But I maintain that it’s (currently) impossible to know if any AI had been inadvertently trained to do the same. So the security implications are no different. If you’ve given an AI the ability to exfiltrating data without any oversight, you’ve already messed up, no matter whether you’re using a single AI you trained yourself, a black box full of experts, or deepseek directly.
But all this is about whether merely sharing weights is “open source”, and you’ve convinced me that it’s not. There needs to be a classification, similar to “source available”; this would be like “weights available”.
It seems like the issue here is, users want to be spoken to in colloquial language they understand, but any document a legal entity produces MUST be in unambiguous “legal” language.
So unless there’s a way to write a separate “unofficial FAQ” with what they want to say, they are limited to what they legally have to say.
And maybe that’s a good thing. Maybe now they need to create a formal document specifying in the best legalese exactly what they mean when they say they “will never sell your data”, because if there’s any ambiguity around it, then customers deserve for them to disambiguate. Unfortunately, it’s probably not going read as quick and catchy as an ambiguous statement.