In order to understand Hume's philosophy of religion, it is crucial to understand the basic tenets of his theory of knowledge. Hume was an empiricist in the tradition of John Locke and George Berkeley; he believed that all knowledge of matters of fact have to come through experience. If you want to know anything about what the world is like, he thought, in other words, you have to go out and investigate; you cannot simply sit in your armchair, think really hard and really well and hope to come up with knowledge. (This might just sound like common sense, but actually it remains a controversial claim among philosophers even today. In Hume's time it was even more controversial, because the 17th and 18th centuries were they heyday of the rationalist philosophers, such as René Descartes, Baruch Spinoza, and G.W. Leibniz, who believed that we could, in fact, arrive at knowledge of some very important matters of fact just by reasoning well, without investigating the world at all.)
Since Hume believed that all matters of fact had to be established through experience, the question of whether religious belief can ever be rational boiled down to the more specific question of whether religious belief can ever be justified by experiential evidence.