I confess, I've been interpreting this phrase like this hapless Chinaman more and more recently, and it slightly irked me every time I heard it. It is, of course, shamelessly self-negating, when the speaker announces that something isn't worth mentioning, but then mentions it nonetheless, sometimes quite elaborately. The Russian не говоря о leaves me with the same vague sense of incredulity.
Modern online dictionaries suggest the complete opposite. Cambridge Advanced Learner's claims that not to mention is a phrase of emphasis rather than of an almost apology for saying something that's already well known to the audience:
