Abstract: Autonomous driving systems have limited the agricultural machinery in complex and unstructured environments. Some challenges still remain, including the high costs of the high-precision sensors, signal susceptibility to the interference from the canopy cover or terrain features, and low environmental adaptability with dynamic or unforeseen conditions. Fortunately, the intelligent transformation is ever-increasing with the rapid advancements in artificial intelligence. The end-to-end (E2E) autonomous driving can be expected to offer a powerful solution to intelligent navigation and precise operations. In this paper, a systematic review was provided on the theoretical adaptability of the E2E autonomous driving. A full-path analysis was also conducted under application scenarios. Literature survey and technical analysis were combined to systematically examine the relevant research from multiple dimensions, including the technological evolution, principles, and typical application scenarios. The key technical challenges were then given under agricultural environments, such as their high degree of dynamic variability, including lighting fluctuations, weather, crop growth stages, and soil conditions; there was a frequent lack of stable navigation references, like permanent lane markings. High precision was often required for the high yield and low resource waste in the agronomic tasks. The principles of the E2E autonomous driving then included the intelligent strategies and development paths. Its framework was integrated with multiple-task learning. Multimodal perception fusion and deep learning architectures were specifically designed for agricultural contexts. A unified neural network was constructed to directly map the sensor data to the control commands, thereby eliminating the information loss among modules. Excellent environmental adaptability was achieved after construction. The low-cost sensors were integrated, like cameras and Inertial Measurement Units (IMUs). A redundant environmental perception was achieved with the advanced network architectures, such as Convolutional and Recurrent Neural Networks. The optimal data after processing fully met the dual demands for low cost and high precision in field operations. Subsequently, a series of typical agricultural scenarios were also extended, including the open fields, orchards, and forestlands, special crop environments, as well as multi-machine collaboration. The E2E approach was achieved in the general pathways and technical advantages. For instance, the E2E system demonstrated superior capabilities on the complex behaviors in open fields, like headland turning and non-linear path following. In orchards and forestlands with complex 3D navigation, Deep Reinforcement Learning-based frameworks have realized robust 3D spatial path planning and maneuverability in GPS-denied areas. In specialized high-value crop environments, the E2E models facilitated the centimeter-level precision and complex environment adaptation for the subtle tasks. In multi-machine collaboration, Multi-Agent Reinforcement Learning architectures were utilized to effectively swarm the intelligence coordination in communication-constrained settings. Key challenges were also identified on the practical hurdles, such as the data scarcity in the agricultural environment, computational resource limitations, complex system integration, and cost-sensitivity in various scenarios. At the same time, the key solutions were summarized, according to the existing research and technological trends, such as simulation-to-real transfer, domain adaptation techniques, and model optimization for edge deployment. Finally, a forward-looking perspective was presented on the integration trend of the E2E technology, particularly with the emerging technologies, like multimodal large models and agronomy knowledge graphs. Intelligent machines were combined with unprecedented reasoning, complex commands, and agronomic contexts. The finding can provide a technical roadmap and theoretical reference for the next generation of cost-effective, highly robust intelligent agricultural machinery. Ultimately, a key driver can also help advance high-quality smart agriculture.