This works presents algorithmic advances and field trial results for autonomous sample return, based on our entry in the 2012 NASA Sample Return Robotic Challenge. The challenge requires finding, collecting and storing pre-defined samples of different colors, shapes and sizes in an 80,000 m2 area in under two hours, without relying on GPS or magnetometer data and with a weight limit of 80 kg for the vehicle. We propose a system level solution to the sample return problem, and makes specific contributions in terms of mapping, planning and sample detection strategies that run in real-time on our custom platform. Field tests demonstrate reliable performance for each of these three main components of the system individually, and high-fidelity simulation based on recorded data playback demonstrates the viability of the complete solution.