The mind can very easily think of worldly objects. It is its Svabhava (=nature). Thoughts generally flow with ease towards objects. Mental energy will readily flow in that direction. The mental force can easily flow in the old grooves and avenues of mundane thoughts. It finds it extremely hard to think of God and dwell on lofty thoughts.
The difficulty in weaning the mind from objects and fixing it on God is the same as in making the river Ganges flow uphill towards its source instead of its natural flow towards the ocean.
Still, through strenuous efforts and renunciation, the mind must be trained to flow towards God, much against its will, if you want to free yourself from birth and death. There is no other way if you want to escape from worldly miseries and tribulations.
(pgs. 116-117, Mind—Its Mysteries and Control by Swami Sivananda)