As varnish doesn’t have any persistent storage you shoul avoid restart it, beside modifications on health checks, backend definitions or changes on the vcl_hash routine.
To reload varnish configuration you could use the distribution init script, or you can do it by hand:
# varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082
vcl.load conf_two /etc/varnish/prod.vcl
vcl.use conf_two
quit
Note that you can use these commands to load multiple setting files and switch between them on the fly.