Effective and Efficient Methods for Constrained Stochastic and Derivative-free Optimization