You don't need planning permission providing that either, your 'permitted development' rights have not been removed by a condition on a previous planning permission or, the structure is not within 5 metres of any boundary.
If any structure is no longer used to shelter livestock, the structure and any hardstanding or other structure forming its base must be removed and the land restored to its former condition.
Bye-laws note: you don't need building bye-law permission to construct a detached building that is used to keep animals, provided it is located so that no part of it is closer than 1.5 times its height to the boundary or any building containing sleeping accommodation, and no part of its interior is more than 18 metres from a fire exit.