A new framework for cross-modal knowledge distillation (CMKD) has been developed, allowing a teacher model to guide a smaller student model without the need for costly paired data. This method improves performance across various multimodal benchmarks by focusing on aligning distributions rather than individual samples.