Mix flour, sugar, baking powder, and salt together. Grate nutmeg on a fine toothed grater or grind it up in a spice grinder and add into the dry ingredients.
Add in olive oil, egg, and milk. Mix gently until most of the lumps are gone.
Peel and remove the core from your apples and chop into very small chunks. Mix diced apples into your mixture.
Preheat a skillet or frying pan to medium heat. Add oil to pan then spread around.
Scoop batter onto hot pan using a ¼ cup measuring cup. Spread batter out just a little so that the apple chunks are not stacked. You will want your pancake batter to be about ⅛ to ¼ inch thick. If it is thicker than that it will not cook very well in the middle.
Cook for about 2 minutes or until bubbles start to form in the pancake and the edges change from shiny to dull. Then flip and cook for about 1 more minute. Pancakes should be golden brown on both sides.
Add more oil to pan as needed and repeat with the remaining batter.
Serve with butter and your favorite syrup or other pancake topping.