Second harmonic generation and sum frequency generation (SHG and SFG) provide effective means to realize coherent light at desired frequencies when lasing is not easily achievable. They have found applications from sensing to quantum optics and are of particular interest for integrated photonics at communication wavelengths. Decreasing the footprints of nonlinear components while maintaining their high up-conversion efficiency remains a challenge in the miniaturization of integrated photonics. Here we explore lithographically defined AlGaInP nano(micro)structures/Al2O3/Ag as a versatile platform to achieve efficient SHG/SFG in both waveguide and resonant cavity configurations in both narrow- and broadband infrared (IR) wavelength regimes (1300-1600¿nm). The effective excitation of highly confined hybrid plasmonic modes at fundamental wavelengths allows efficient SHG/SFG to be achieved in a waveguide of a cross-section of 113¿nm¿×¿250¿nm, with a mode area on the deep subwavelength scale (¿2/135) at fundamental wavelengths. Remarkably, we demonstrate direct visualization of SHG/SFG phase-matching evolution in the waveguides. This together with mode analysis highlights the origin of the improved SHG/SFG efficiency. We also demonstrate strongly enhanced SFG with a broadband IR source by exploiting multiple coherent SFG processes on 1¿µm diameter AlGaInP disks/Al2O3/Ag with a conversion efficiency of 14.8%¿MW-1 which is five times the SHG value using the narrowband IR source. In both configurations, the hybrid plasmonic structures exhibit >1000 enhancement in the nonlinear conversion efficiency compared to their photonic counterparts. Our results manifest the potential of developing such nanoscale hybrid plasmonic devices for state-of-the-art on-chip nonlinear optics applications.