My father passed away 3 months ago. My mother opted for the usufruct. My brothers and I are liable for inheritance tax. I'd like to ask the tax department to defer payment until the day the usufruct ends. Should my brothers also request this ?

No. You may be the only one to opt for deferred payment of inheritance tax. 
However, it is imperative that your brothers give their express agreement to this request, and that they declare that they have been informed that, in the event of default on your part, they remain liable to pay the inheritance tax due to the joint and several liability between you (art. 1079 CGI), even if they have already paid their own tax (BOI-ENR-DG-50-20-40 §120).
If one of them refuses to give authorization, your request will be rejected by the accountant (BOI-ENR-DG-50-20-30 §120).