This is because [tex]\tan x[/tex] is a multivalued function and is not invertible over its entire domain. We restrict its domain to the interval [tex]-\dfrac\pi2<x<\dfrac\pi2[/tex], which gives one complete branch of values (or one period). [tex]\dfrac{4\pi}5>\dfrac\pi2[/tex] and thus [tex]\dfrac{4\pi}5[/tex] is outside the domain.
A different way to go about this is to find the value of [tex]\tan\dfrac{4\pi}5[/tex] first, then compute the inverse tangent of that result. But finding the trigonometric values of multiples of [tex]\dfrac\pi5[/tex] is somewhat tricky and perhaps more work than is needed.
Instead, we can use a trigonometric identity to find the value of [tex]\tan[/tex] whenever its argument falls outside the "standard" branch.
We know that [tex]\tan(x\pm\pi)=\tan x[/tex] (because [tex]\tan[/tex] is [tex]\pi[/tex]-periodic), so [tex]\tan\dfrac{4\pi}5=\tan\left(\dfrac{4\pi}5-\pi\right)=\tan\left(-\dfrac\pi5\right)[/tex]. And now the function can be inverted, so that
[tex]\tan^{-1}\left(\tan\dfrac{4\pi}5\right)=\tan^{-1}\left(\tan\left(-\dfrac\pi5\right)\right)=-\dfrac\pi5[/tex]