One must break one’s fast at the correct time. The following story from the scripture underline this:
“The King said: ‘To transgress the laws of respectful behavior toward the brahmanas is certainly a great offense. On the other hand, if one does not observe the breaking of the fast within the time of Dvadasi, there is a flaw in one’s observance of the vow. Therefore, O brahmanas, if you think that it will be auspicious and not irreligious, I shall break the fast by drinking water.’ In this way, after consulting with the brahmanas, the King reached this decision, for according to brahminical opinion, drinking water may be accepted as eating and also as not eating.” Srimad Bhagavatam 9:4:39-40
By not breaking the fast in time it creates a blemish on the fast; a blemish so serious that Maharaja Ambarish was put into a dilemma as to how to solve it. If it were something minor then why this whole drama which resulted in Durvasa wanting to kill him and so on? So it would seem that the blemish incurred was serious enough that Ambarisha asked his ministers what to do and risked the curse of Durvasa in order to break the fast on time. Just eating even one grain of rice is a blemish sufficient to ruin the whole Ekadasi vrata. Similarly not breaking the fast within the appointed time is a serious enough blemish that Ambarish risked death (brahma-sapa) in order to do so. Hence, it would be incumbent on us all to take this into consideration and breakfast at the correct time.