The optimization of membership function parameters in Fuzzy-PID controllers presents significant computational and design challenges, often relying on time-consuming and heuristic-based methods that lack generalizability. In this paper, we present a novel and efficient technique to enhance the performance of Fuzzy Inference Systems (FIS) by employing Deep Reinforcement Learning (DRL) for automatic and intelligent parameter tuning. Fuzzy logic offers significant advantages in handling nonlinear, uncertain, and highly dynamic systems due to its rule-based reasoning and flexibility. Meanwhile, DRL has demonstrated outstanding capabilities in learning optimal control strategies in complex environments through trial-and-error interactions and policy optimization. By combining the strengths of both paradigms, our proposed method enables the automatic adjustment of membership function parameters without manual tuning, leading to improved control accuracy and system adaptability. The proposed DRL-Fuzzy PID framework is specifically applied to the control of Unmanned Aerial Vehicles (UAVs), which are highly nonlinear systems that demand robust, adaptive, and precise trajectory tracking. Additionally, by strategically reducing the number of parameters involved in the optimization process, we significantly shorten the learning time and reduce computational overhead. Extensive UAV simulation studies confirm the robustness and efficiency of the proposed approach, showing substantial improvements in tracking performance and overall flight stability compared to traditional fuzzy controllers. This hybrid DRL-Fuzzy strategy offers a promising solution for advanced UAV control applications where adaptability and precision are critical.