Dario Amodei
- Handle: @DarioAmodei
- Title: —
- Posted: —
- Source: https://x.com/DarioAmodei/status/2088758816376807762
> I don't usually spend much time on social media but I wanted to engage here because it really brings out the heart of an important conversation. First, on regulation, I think that “either concentrate it in the hands of a chosen few companies and politicians via regulation or distribute it widely” is a false choice. I know that there's a sort of Silicon Valley shorthand where regulation = regulatory capture = concentration of power, but I've always found this to be an overly simplified picture of the world. Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people. I don't necessarily agree with that perspective either, rather I think it's complicated and really depends on what the "regulation" consists of. But in particular I think that those in the "regulation = regulatory capture = concentration of power" frame often underrate the decentralizing power of objective and fair institutional processes. A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice. At their best, institutions can vest power in ideas rather than people, and thereby decentralize that power. This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage (slow down) frontier AI companies while *advantaging* smaller competitors. California's SB53 (which we supported), and even the much-maligned SB 1047 (which we were ambivalent on), completely exempt any company below a certain amount of revenue or model training costs from being covered at all (it was $500M for SB 53, lower for 1047 but we objected to that). More recently the testing process we've advocated for at CAISI and the White House involves more rigorous tests for frontier models than off-frontier models — something that differentially advantages challengers. Similarly, the "Pacing the Frontier" letter envisions (or at least Anthropic's preferred implementation of it envisions) modulating the pace of the very best models while not constraining those who are catching up. This hurts the business interests of the frontier labs and helps challengers, including open-weights! Overall my view is that AI is *structurally* a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws). Open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips (which are roughly the frontier labs plus maybe hardware providers). By contrast I think the right "rules of the road" can simultaneously (a) address AI's cyber/bio/alignment risks, (b) institutionally constrain the power of the frontier AI companies, and (c) leave room for open-weights models while also addressing the specific risks that they bring.
中文要旨： 我通常不會花太多時間在社交媒體上，但我希望在這裡參與，因為這真正揭示了一個重要對話的核心。首先，關於監管，我認為「要麼通過監管集中在少數公司和政治家手中，要麼廣泛分布」是一個錯誤的選擇。我知道硅谷有一種簡化的說法，即監管=監管捕獲=權力集中，但我一直認為這是一種過於簡化的世界圖景。許多在這個圈外的人將監管視為限制企業權力並造福普通人的東西。我也不一定同意這種觀點，而是認為這很複雜，真正取決於「監管」的內容。但特別是，我認為那些在「監管=監管捕獲=權力集中」框架中的人往往低估了客觀和公平的制度過程的去中心化力量。一個粗略的類比是，正式的司法繫統有時會感覺沉悶和精英主義，但它對保護脆弱個人權利的工作比替代方案 mob justice 做得更好。在最好的情況下，機構可以將權力賦予思想而不是個人，從而去中心化權力。這就是為什麼 Anthropic 一直非常小心地制定其政策建議。我們努力使建議對前沿 AI 公司不利，同時「有利於」較小的競爭對手。加州 SB53（我們支持），甚至被廣泛貶低的 SB 1047（我們持中立態度），完全豁免任何收入或模型訓練成本低於一定水平的公司，使其完全不受覆蓋（SB 53 的門檻是 5 億美元，SB 1047 的門檻更低，但我們反對）。最近我們在 CAISI 和白宮倡導的測試程序涉及對前沿模型進行更嚴格的測試，而非前沿模型——這對挑戰者有差異化優勢。同樣，「調整前沿」信件預設（或至少 Anthropic 偏好的實現方式）調節最優秀模型的速度，而不限制那些正在追趕的模型。這損害了前沿實驗室的商業利益，並幫助挑戰者，包括開放權重！總的來說，我的觀點是，AI 是一種*結構性*技術，傾向於集中權力，原因與監管無關（更多與規模定律的極端影響有關）。開放權重確實有助於某些方面，但遠遠不夠，因為它只是將集中程度轉移到那些擁有最多計算能力和晶片的人身上（大致是前沿實驗室加上可能的硬件提供商）。相反，我認為正確的「道路規則」可以同時（a）解決 AI 的網絡/生物/對齊風險，（b）制度上限制前沿 AI 公司的權力，（c）為開放權重模型留下空間，同時處理它們帶來的特定風險。
三個爭議點
1. Critics may argue that Amodei's stance on AI regulation is too industry-friendly, potentially overlooking long-term societal implications.
2. His discussion on 'open-weights' models may be seen as a way to justify the dominance of large tech companies rather than truly democratizing AI.
3. The complexity of his regulatory views might be perceived as an attempt to obfuscate the core issue of AI governance and power concentration.