Whip the cold heavy whipping cream on medium speed just until stiff peaks form; about 2-3 minutes. Set the bowl aside.
In a separate bowl, warm the peanut butter for about 30 seconds until melted and thin. Add the sweetened condensed milk and vanilla extract to the bowl and mix until combined.
Gently fold the whipped cream and peanut butter mixture together until totally combined. Fold in the chopped nutter butter cookies, reserving some for the top.
Pour ice cream mixture into a 9x9 or large loaf pan and spread into an even layer.
Heat the remaining peanut butter for about 20-30 seconds in the microwave until melty. Drop dollops of the peanut butter all over the top of the ice cream. Use a butter knife or chopstick to swirl the peanut butter dollops into the ice cream. Sprinkle on more chunks of nutter butter cookies, if desired.