Sam is telling friends, there's no way Brit's ex-nanny Leah Frand could have known that the singer had crystal meth in the house ... because she was fired after only one week.

Sam's claims contradict what Leah told us yesterday ... that she worked for Britney for 6 months back in 2007 -- during the crazy meltdown period -- and never once saw any trace of drugs.

But Sam's saying that's not true -- telling friends Leah was quickly canned for stealing one of Britney's dresses (which she later returned) ... so her opinion counts for exactly zero.

According to sources, Sam also wants to clarify one thing -- he never said Britney did crystal meth, only that he found a stash of crystal meth in her house.