Apart from direct quotes, is there ever a time when I'd use a page number only?
Answer
I'm a little confused by your question, but I will answer it based on what I think you mean.
The essential parts of a parenthetical reference are author & date of publication. These let the reader find the full reference in your works cited page.
The page number is added when you directly quote your source material.
Depending on how you write your sentence, and quote your material, it is possible to only have the page number in parentheses, but the page number cannot exist with the author & date of publication.
See the first example in the "In-Text and Parenthetical Citation Examples" box on this Library page - http://libguides.css.edu/content.php?pid=61826&sid=454619
I cannot image a scenario when you would be discussing something on a page with the context of what the document the page came from.
I hope I got to what you were asking.