Large language models struggle with causal discovery due to fundamental limitations in their learning paradigms, which prevent them from distinguishing between similar causal graphs. The proposed Agentic Causal Bayesian Optimization (A-CBO) offers a promising alternative by using a frozen language model to effectively analyze intervention effects without the need for extensive retraining.