I stacked rocks in such a way to make holes and hiding places for my fish in my concrete pond. I only did this in some areas. I picked out pretty rocks. Which is funny to me now since all are covered in the same green black algae and the original rock color is lost.
If you put in pea gravel it does give healthy pond bacteria more surface area and it gives fish something to root around in. Eventually you will have to clean some muck that accumulates and it will be hard not to remove pea gravel with the muck. No problem, You can save some and add new.
I did not do pea gravel on the bottom, but some has flowed in from my bogs.
I think it is a matter of personal taste. Your entire pond will turn a natural color as algae grows on everything.
With my new pond I planned fish caves so I do not need the stacked rocks. But I will be putting rocks in to hide where the tubing and air hose go, so there will be added rocks just over the piping, not on the entire floor of the pond.
Also keep in mind what kind of footing the rocks will be. You may say you will not walk in a 4 foot pond.............maybe not. But maybe you will need to walk in it for some reason even if you partially drain it first.
Are you doing koi or goldfish?