Reuse: Remove only external references when adding to library

When removing references before adding elements to a library, only
references that leave the elements containment (sub-)tree are removed.

Issue-ref: 4346
Issue-URL: af3#4346



Signed-off-by: Andreas Bayha <bayha@fortiss.org>