POLAR-Bench is a new benchmark designed to evaluate the privacy and utility trade-offs of large language model (LLM) agents when handling user data. The findings indicate that while advanced models excel in protecting user privacy, smaller models commonly used by individuals tend to leak significant amounts of sensitive information.