Some will be tempted to say: if mathematical objects figure in the explanation of the motions of physical objects in the universe (e.g., the neurons in human brains), then it follows trivially that they are themselves physical, and not abstract. The move is common among naturalists. John Searle, for example, says that
For us [naturalists], if it should turn out that God exists, that would have to be a fact like any other. To the four basic forces of the universe—gravity, electromagnetism, weak and strong nuclear forces—we would add a fifth, the divine force . . . [I]t would still be all physics, albeit divine physics. If the supernatural existed, it too would have to be natural. [Searle, 1998, p. 35]
This sort of terminological appropriation, whether it is applied to God, numbers, or anything else, fails to address the underlying question. By decreeing that the word ‘natural’ (or ‘physical’) is to be applied to any phenomenon we discover, the naturalist robs naturalism of any content relevant to the substantive dispute between naturalists and those who disagree with them. I have claimed that efficient causal relations between non-spatial, necessary, eternal, unchanging objects and spatial, contingent, changing objects are strongly possible, and I have used the word ‘abstract’ to refer to the former sort of objects, and ‘physical’ or ‘material’ or ‘concrete’ for the latter sort. But the truth of my claim is not affected, or illuminated, if we decide to use these words in some other way instead.
-- Callard, Benjamin, The Conceivability of Platonism, Philosophia Mathematica (III) 15 (2007), pp. 347–356.