Bird outside my window...?

HI ... I have a bird nest located in a cedar shrub just outside my bedroom window.... I lived thru this last year .... Mornings with NO SLEEP.... It is a VERY NOISY Black Bird....Would it be wrong of me to remove the nest right now, before they lay eggs???? I have checked and it IS empty... There are many trees around so I am SURE they would find a new home. What do You Think? Thanx.