Engineering Low-Loss Optical Interconnects for Rare-Earth Quantum Memory