All you'll have to gain is points.
This is correct. As you say, is pertains to the "that which you'll have to gain", not to the points.
Consider the following sentence:
All you have to gain is your voice and overall health.
This is correct. Using the word are there would clearly be wrong. This is because is pertains to "all you have to gain", not the two objects that follow.
How to prove it? Alright, try a web search for "have to do is" (in quotes), and you get 136,000,000 results. A look at the list shows the sort of construct we have been discussing here.
Then, try googling "have to do are" (again in quotes). For one thing, you get much fewer results. But more importantly, if you look at the results you won't see any that are the type of construct we have discussed. Instead, you'll see sentences like:
The main things you have to do are:
So here the noun (things) are is referring to is already pluralized. That's why we have are instead of is.
Look through the results in the second Google link. You won't find any uses of are in the type of sentence you refer to. (Or, if you find one or two, it will be by someone using faulty English.)
Finally, I think the whole sentence is a bit weird. I would probably post a new question asking if the sentence itself makes sense in context.