The Furies
The Furies (not to be confused with the Furries), are a group of quasi-goddesses representing vengeance and punishment for oath-breakers. They were originally created when Cronus ripped off his dads balls and threw them into the ocean. The testicular blood fell upon the earth (Gaea) and reformed into the furies. The number of them varies, but there are three known for sure:
Alecto (Meaning Endless)
Megaera (Meaning Jealous rage)
and Tisiphone (Meaning Vengeful destruction).
Pleasant bunch, right? In Greek myth, they typically waltzed around punishing the snot out of people who the gods considered to be reprehensible. Considering how horribly flawed the Greek pantheon is, it shouldn't come as a surprise that the punished were just as often innocent as guilty.
The Furies are considered true neutral, and will mete out justice on any and all who break their oaths, for any reason. They aren't typically worshiped in the traditional manner, though they are venerated in murals in places like courthouses. Are your players being massive, raging assholes? Toss the Furies at them. In older editions, their whips permanently drained stats.
For the far less powerful devil they inspired in Dungeons & Dragons, and their Ravenloft namesakes, see Erinyes.