I'm in the camp saying contact the client before proceeding with the appraisal. Until you do that, you don't know what their lending policy/guidelines are, or what they may be concerned about.
In certain cases, contact with legal authorities may be warranted.
You probably will be asked to be very specific as to what you observed. In some cases, you might even be asked to provide exterior photos at the very minimum to reveal what was observed.
A vast majority of our assignments are for mortgage lending purposes. As such, we appraisers are the eyes/ears for the lender. I'm constantly amazed by the number of responders who ignorantly recommend appraisers just appraise the real estate and disregard 'illicit' or 'unlawful' activity that may be in evidence on the property. Doing so has the potential to hide fraud or crimes that may be occurring. The appraiser then becomes complicit.
One really must be willing to turn inward and examine these kinds of situations ethically. Decline any assignment where theire is any hint of gut churning, and let some other person deal with it.