Using a mixer, beat cream cheese and pumpkin puree until smooth. Add powdered sugar (½ cup at a time), pumpkin pie spice, and ginger until combined. If your dip is too thick, add 1 – 2 tablespoons of heavy cream to thin it out. Keep in refrigerator and serve cold.