Only the assignments in the "Upcoming" category, which are not visible to students yet, can be deleted. At the moment it is not possible to delete assignments in the "Active" and "Completed" categories. Even if you delete the users or groups they were assigned to, the assignments will not disappear.