A new framework called HAM$^{3}$ has been developed to explore vulnerabilities in multi-modal multi-agent systems (MM-MAS) through hierarchical attacks. The framework reveals significant weaknesses, achieving an Attack Success Rate of up to 78.3%, highlighting the need for improved robustness in these systems.