The trouble with all rational demonstrations of the existence of God, with which the history of philosophy abounds, consists in their being exactly what they were meant to be by those who formulated them: abstract logical demonstrations divorced from the living primal experiences in which these demonstrations are rooted. For instance, the cosmic experience was transformed into a cosmological proof, the ontic experience into an ontological proof, et cetera. Instead of stating that the most elementary existential awareness as a subjective "I exist" and an objective "the world around me exists" awareness is unattainable as long as the ultimate reality of God is not part of this awareness, the theologians engaged in formal postulating and deducing in an experiential vacuum. Because of this, they exposed themselves to Hume's and Kant's biting criticism that logical categories are applicable only within the limits of the human scientific experience.Click here to read moreIn an earlier essay, The Halakhic Mind (p. 118 n. 58), he wrote that "[t]he problem of evidence in religion will never be solved."
In an interesting analysis, R. Shubert Spero turns the quote from The Lonely Man of Faith on its head and demonstrates that R. Soloveitchik was actually proposing a proof of God's existence, essentially a restatement of Max Scheler's approach ("Rabbi Joseph Dov Soloveitchik and Belief in God" in Modern Judaism 19  and summarized in this lecture: link).
Scheler addressed the age-old problem of how we know that we exist. After Descartes and Kant, it is difficult to conclusively establish that we or the world exist. However, despite this we still accept that we and the world exist because we experience it. Scheler takes this further and states (Scheler, On the Eternal, pp. 257-258):
Just as the existence of an external world or the existence of a unitary self cannot be proven and are considered primary data albeit asensual and as present before the sensations transmitted via the sense of a living being--so, too, the sphere of the phenomena disclosed to the mind by the religious act, the sphere of the divinity, and of the general reality it contains is a primary datum which it is impossible to derive from anything else... Hence there can be no question of proving the existence of the entire religious sphere by inference from other facts of the world, as little question as there is of "proving" the existence of the external world, of the self or of one's fellow creatures. To insist on such proofs is to misconstrue in principle the limitations of proofs and the extent of their validity.In other words, because we experience ourselves, other people and the world, we accept that experience as reasonable evidence of their existence. Similarly, we should take our experience of God as sufficient evidence of His existence. Compare this with R. Soloveitchik's words:
Instead of stating that the most elementary existential awareness as a subjective "I exist" and an objective "the world around me exists" awareness is unattainable as long as the ultimate reality of God is not part of this awareness...R. Spero adds two conditions to this type of experiential evidence: the experience is compelling and cannot be ignored, it is not contradicted by further experience. R. Spero emphasizes that this is not a proof of God in the conventional sense but a justification of belief in God. In explains how belief in God is reasonable to a rational person. However, this approach is entirely dependent on a person's experience and only an individual can make a judgment for himself on the subject.
But how do we know that this experience is real and not a figment of our imagination? Prof. Abraham Joshua Heschel addressed this in his Man is Not Alone, pp. 81-84:
The ultimate question in its logical form is an ever-present challenge which we encounter wherever we turn, and there is no way of ignoring it. Man cannot afford to be noncommittal about a reality upon which the meaning and manner of his existence depend... In whatever decision he makes, he implicitly accepts either the presence of God or the absurdity of denying it. The nonsense of denial is too monstrous to be conceivable...
If the divine were a complex notion, then we might have suspected it to be a product of fancy, a combination of characteristics found separately in the world and which are imagined to exist together in one being. But the divine as a first insight is a reality, transcending both the power of mind and order of the world rather than a compound of characteristics found within the world.
The divine is too ineffable to be a product of the human mind, too grave, demanding and all-surpassing to be postulated by wishful thinking. Where would such an awareness come from, if not from an underivable insight into His all-surpassingness? The question, however, may be asked: Don't we often cherish beliefs which are afterwards exposed as delusions? Yes: we may believe that we see a house when driving through the desert and upon trying to reach it, it may turn out to be a mirage. But we cannot think that a picture represents a house if there is no such thing as a house.