The BEAMS Initiative aims to develop responsible AI tools for modeling and simulation by establishing benchmarks that prioritize human-centered practices. Initial evaluations show that while AI tools excel in qualitative tasks, they struggle with causal reasoning and quantitative error correction, highlighting the need for ongoing improvements and considerations of bias.