This paper presents an application of the nonlinear optimal control techniques to the design of launch vehicle autopilots. The optimal control is given by the solution to the Hamilton-Jacobi-Bellman (HJB) equation, which in this case cannot be solved explicity. A method based upon Successive Galerkin Approximation (SGA), is used to obtain an approximate optimal solution. Simulation results involving three degrees of freedom (3DOF) model of a launch vehicle during atmospheric flight are reported to demonstrate the performance of the considered autopilot. Alternative results are also presented for classical and linear optimal approaches.