In formal language you can use "penultimate" and "antepenultimate" repectively. "Second to last" and "third last" are options usable in more usual language.
(OALD) penultimate adjective /pəˈnʌltɪmət/ [only before noun]
immediately before the last one
synonym next/second to last
"Second to last" is neither formal nor informal, it's neutral.
This compound word can be used as an adjective, as shows the ngram below.

There are also the shorter forms "second last" and "second-last".

The hyphenless version is more common; the cases for "line" and "paragraph" seem to show that.

The form "last but one" can also be used but it is less common.

(OALD antepenultimate adjective /ˌæntipenˈʌltɪmət/[only before noun] (formal)
two before the last; third last
• the antepenultimate item on the agenda
"Third last" (neutral term) can be used as an adjective.

The hyphenless version is more common; the cases for "line" and "paragraph" seem to show that.

An ngram of comparison between "antepenultimate" and "third last" shows that in fact the first is used nowadays only slightly more frequently.
