While setting up blocked terms and phrases I accidentally didn't set the term to private. When I went back to correct this the private terms were still sitting in my "Mod Action" panel. I assumed it would work retroactively, but it didn't and it needs to.