How do you make peanut butter brownies with Reese's Peanut Butter Cups? Place a Reese's peanut butter cup upside down on top of the cookie dough. Evenly distribute the brownie batter on top of each peanut butter cup until the wells are about 90% full. Bake for 18-20 minutes or until the brownie batter is cooked through.