You're trying to minimize the distance between the point [tex](18,0)[/tex] and an arbitrary point on the curve, [tex](x,y)=(x,x^2)[/tex].
The distance between two such points is given by the function
[tex]d(x)=\sqrt{(x-18)^2+(x^2-0)^2}=\sqrt{x^4+x^2-36x+324}[/tex]
so this is the function whose derivative you should check.
But before you do that, it's helpful to know that [tex]d(x)[/tex] is minimized at the same point as the modified distance function [tex]d^*(x)=d^2(x)=x^4+x^2-36x+324[/tex].
Differentiating, you have
[tex]\frac{\mathrm d}{\mathrm dx}d^*(x)=4x^3+2x-36[/tex]
Set this to zero and solve for the critical points.
[tex]4x^3+2x-36=2(2x^3+x-18)=0[/tex]
You can use the rational root theorem to find some potential candidates for roots to the cubic. The constant term has factors [tex]\pm1,\pm2,\pm3,\pm6,\pm9,\pm18[/tex], while the leading coefficient has factors [tex]\pm1,\pm2[/tex]. The only candidates for rational roots are [tex]\pm1,\pm\dfrac12,\pm2,\pm3,\pm\dfrac32,\pm6,\pm9,\pm\dfrac92,\pm18[/tex]. The only one of these that works is [tex]2[/tex], so [tex]x=2[/tex] is a root to the cubic above.
Polynomial division reveals that we can factor the cubic as
[tex]2(x-2)(2x^2+4x+9)=0[/tex]
which has only one real root at [tex]x=2[/tex]. Checking the value of the derivative of [tex]d^*[/tex] to the left and right of this point confirms that a minimum occurs here.
Therefore the closest point on the curve to [tex](18,0)[/tex] is [tex](2,4)[/tex].
