SentinelBench introduces a new benchmark for evaluating AI agents designed for long-running monitoring tasks, emphasizing the importance of sustained attention over continuous action. This open-source tool allows for performance comparison across various web environments, highlighting the tradeoffs between responsiveness and resource use.