Public Works and Infrastructure Minister Patricia de Lille has been accused of wrongdoing relating to a border fence between South Africa and Zimbabwe.