Richard Michael John Ogilvie Graham (Conservative Party), Minister of Parliament for Gloucester