Preheat oven to 350F. Spray a 2 quart baking dish with oil.
Put diced king cake in the prepared dish, making sure the surface is covered.
In a large mixing bowl, whisk together eggs, heavy whipping cream, milk, and vanilla until well combined. Pour this mixture over the diced king cake and use a spatula to gently press the king cake down so everything is covered.
Bake for 30 minutes; cover with aluminum foil and bake another 15 - 20 minutes, or until the custard is set.
Serve hot with vanilla ice cream on top.