ng-click stops working after the first use of $compile when using nested directives

I have an Angular modal directive that uses a helper/wrapper directive. This way I can always use the same wrapper and just load a different template where needed for different modal content.

PROBLEM: This snippet works, but only for the first life cycle of the modal. So I can fire the modal, close the modal and fire it again. But once the modal is open the second time none of the ng-click directives work. Any tips would be just super.

Usage

<button my-modal="views/login.html">Launch Login-specific Modal</button>

Directive Module (app.js)

angular.module('myModal',[])

.directive('modalWrapper', function(){
  return {
    replace: true,
    templateUrl: 'views/modal.html',
    controller: function($scope, $element){
      $scope.close = function(){
        $element.remove();
      };
      // NOTE: I use this array to showcase that ng-repeat still works the second time although ng-click stops functioning properly.
      $scope.others = ["One", "Two", "Three"];
    }
  }
})

.directive('myModal', function( $compile){

      function link(scope, element, attr){

      scope.partial = attr.myModal; // NOTE: Loads sub template via ng-include

      var ngModal = $compile('<div modal-wrapper></div>')(scope);

      element.on('click', function(){
        angular.element('body').append(ngModal);
      });

      scope.yo = function(){
        alert("Yo from inside template.");
      };

    }

    return {
      link: link,
      scope: {}
    }
});

Templates

modal.html

<div class="my-modal">
    <p>Modal Wrapper</p>
    <div ng-include="partial"></div>
  <button ng-click="close()">Close</button>
  <p>This just proves that other directives still work (ng-repeat), but ng-click does not.</p>
  <div ng-repeat="stuff in others">
    <p>{{stuff}}</p>
  </div>
</div>

login.html

<h1>Well hey there, I'm the login template.</h1>
<button ng-click="yo()">Say Yo</button>