The mirrors are a bit of a ratbag!! The nut with the taper is the only one you need to move!!! It will need backing off a long way then just give a good tug.
I had to do mine after I took them off and had too soak them with WD40 for agess dont forget the right hand mirror has a left had thread as I found out after strugling like a tw*t