Preheat the oven to 350 and prepare a loaf pan with grease or parchment. Set to the side.
Prepare the apples. Peel, core, and cut the apples in large chunks. Toss the cut up apples with the brown sugar, cinnamon, and nutmeg. Set to the side.
In a medium-sized bowl, mix together the flour, baking powder, baking soda, and salt. Set to the side.
In a large bowl with a hand mixer or in the bowl of a stand mixer with a paddle attachment, cream the butter with the sugar until light and fluffy, about 2-3 minutes. Scrape down the sides of the bowls and add the eggs one at a time, followed by the vanilla. Scrape down the sides of the bowl if needed and add in the sour cream and milk. The mixture will look soupy, but that is okay.
Add the dry ingreidents to the wet ingreidents, in two increments. Make sure the batter is fully mixed together. It will be very thick.
Make the crumb topping. Melt the butter in a heat-proof bowl. Combine the melted butter with the cinnamon, brown sugar, and flour. Use a fork to stir the mixture all together. This will allow the crumbs to form.
Assemble the cake. Take the loaf pan and add half of the batter to the pan. Then, add in all of the apples, followed by the remaining batter of the cake. Add the crumb topping to the top layer of the cake, covering all of the batter in the process.
Bake the cake for 55-65 minutes, until a toothpick comes out clean. Let the cake cool fully in the pan on a cooling rack.
