Everybody's language experience is different, so That's not my strong suit would be a transparent metaphor from a card game like bridge or whist to anyone familiar with those games, calculably understandable by others from the sense of strong alone, and probably incomprehensible to still others. There is no single word that describes all of these attributes, sorry; in fact, that's a lot to expect of one word.
Since I learned bridge in college 50 years ago, all my memories of the use of strong suit are bridge-colored, and it makes perfect sense as a metaphor. Indeed, a familiarity with the actual source of a metaphor -- in this case, table talk, and the strategies and tactics that it typically accompanies -- can make for a more nuanced understanding on occasion, I think.
As to the more general question, practically every expression is hard to understand, outside its actual context. People leave off things that they expect everyone to know. And they're often wrong, especially in writing, where one frequently knows nothing of one's readers.