I did an experiment with this, as I read on this forum about putting plants directly in gravel or substrate.
I have a bed of over 100 iris that is in clay. I scooped a few iris out with ample roots and put them in pea gravel earlier this year. The ones in clay are ~1m50 now and blossoming. The ones in pea gravel are less than half their height and do not look too good at all.
Likewise for my lilies. The ones in clay are spawning about 10 pads each with more heading to the surface. The only one I put in a basket with rock substrate has 2 anaemic pads.
Now for iris, Im willing to believe that once their root system matures, they will do okay in gravel too, though not as good as in clay. Lilies, I seriously doubt.
Anyway, to answer your question, as mentioned above dont use regular pot soil, it will wash out and make a mess. I found that the soil they sell as "pond soil" is really no better in that regard, its just more expensive and makes an equal mess. Use clay (kittly litter) or substrate depending on plant. Perhaps a mix of both is even better, something Im gonna try now.