The Masquerade is an organized campaign enforced by the Camarilla to convince people that vampires do not exist. The Masquerade is the cornerstone survival strategy for Kindred, without it, the kine would rise up and exterminate all the undead.
The Masquerade is technically outlined in the Traditions, however until the 15th century this was commonly interpreted as not informing mortals of the niceties of vampiric society, how common vampires were or generally making too big a noise. The Tradition was openly flouted by several clans (notably the Tzimisce), while the vampires in more civilized society would often adhere to it more tightly.
All of this changed with the Inquisition, following the Anarch Revolt, when the true number of vampires became apparent and Rome started to have at the undead igne ferroque, the nascent Camarilla was primarily organized around promulgating and strongly enforcing a Masquerade. History holds that Rafael de Corazon's famous speech in 1450 made enforcing the Masquerade the Camarilla's primary policy.
The basics of the Masquerade are enforced through self-policing and harsh penalties. When a breach (colloquially, an "oops") occurs, the methods of repairing may vary, but somebody is going to be punished — and while it would be nice for that somebody to be whoever caused the breach in the first place, Princes are known to prefer an early morning dusting over a long thorough search.
Generally, a breach consists of any evidence of vampiric powers: feeding on blood, obvious use of Disciplines, regeneration in plain sight and the like. Certain things (such as ghouls) are considered potential breaches all the time; although in this case, its assumed that the domitor will take necessary steps.
Camarilla vampires are taught to blend in and to cultivate their Humanity in order to make imitating living people easier. However, when something severe happens, the Camarilla usually calls on the Ventrue, who have made Masquerade-patching something of a science. The Ventrue will apply whatever resources are necessary; cash, judicious use of Dominate, machine guns; to make the problem go away.
While the Sabbat does not itself have a Masquerade, it survives primarily through the Masquerade enforced by the Camarilla, and a similar "patch squad" managed by the Grimaldi ghoul family.