Preheat the oven to 375 degrees. Coat a muffin tin with cooking spray.
Combine the flour, sugar, baking powder, baking soda, and salt together in a bowl. Mix until well combined.
Whisk together the lemon yogurt, zest, and juice from the lemon, melted butter, egg, milk, and vanilla together until well combined. Add the flour mixture to the yogurt mixture and stir until just combined.
Pour the batter evenly into the muffin tins. Place into the oven and bake for 25 minutes, or until a tester inserted in the middle of a muffin comes out clean. Remove from the oven and let them sit for 5 minutes before moving them to a cooling rack.
Make the glaze by adding the lemon juice to the powdered sugar and mix until well combined. Once the muffins are cool, place a piece of wax paper under the cooling rack with the muffins (for easier clean up), then drizzle the glaze over top of each muffin evenly. Serve immediately. Enjoy.