The version "my playing chess" has the gerund "playing" and the direct object "chess". This is okay, since a gerund is a verb, and a transitive verb like "play" can take a direct object.
The second version, "my playing of chess", may be acceptable (it doesn't sound as good to me), but it's not a gerund. The fact that the logical object "chess" is preceded by "of" indicates that "playing" is a noun. This is what happens to logical objects of nouns, since grammatically, nouns cannot take direct objects. "Chess" has to be converted to a prepositional phrase because of this grammatical requirement. (Compare the verb "father" with the direct object "a son" and the noun "father" with the prepositional object "of a son".)
So, since in "playing of chess", the "playing" is a noun, it can't be a gerund (which is a verb form). This doesn't mean there is anything wrong with it, because there is a derivational suffix "-ing" which creates nouns from verbs, so "playing" may be a noun created by the addition of this suffix to the verb "play".
From the Wikipedia article on Gerunds:
- I like playing football. (playing takes an object, so is a gerund)
- Her playing of the Bach fugues was inspiring. (playing takes a prepositional phrase rather than an object; not a gerund)
playingdoesn't function as a gerund in it. The -ing form can function as a gerund, a participle and a verbal/deverbal noun. Gerunds take an object. The secondpayingtakes a PP, ergo, not a gerund. – Tushar Raj Jul 09 '15 at 20:23