pw-metadata -n settings 0 clock.force-quantum 2048
The low latency of pipewire is what results in this. The above command will temporarily increase it. The Debian Wiki has more details and instructions on how to make the changes permanent if you wish.