When up-selling a similar or an upgrade from the target product, you may enable the Replace button. 



However, if you prefer your customers to simply add items to cart, you may change the Call To Action button from "Replace" to Add-to-cart. Simply uncheck this option in your upsell offer settings:


 


Very simple and you're done.