Unless the OP's Zoning ordinance contains verbiage similar to...
(I have no desire to read ordinances from the left coast but this is from a local Zoning ordinance)
"12.10 Improvement Location Permit
a. Improvement Location Permits Required
.
No building or other structure shall be erected, demolished, moved, added to, or altered, changed, placed, or be established or changed in use, on platted or unplatted lands without a permit issued by the Director..."
"h. Failure to Obtain an Improvement Location Permit
Failure to obtain an Improvement Location Permit shall be a violation of this Ordinance and punishable under Chapter 13: Violations and Enforcement."
"13.2 ZONING VIOLATIONS
A. DESCRIPTION OF A VIOLATION
1. Whoever violates any of the provisions below shall be guilty of an ordinance violation. Each day during which a set of facts exists that constitutes a violation or offense, it shall
constitute a separate offense.
...
b. Failure to obtain the necessary Improvement Location Permit, or failure to obtain the necessary certificate of occupancy or any other necessary permit as established in this Ordinance."...
As I read it, failure to obtain a permit constitutes a zoning violation, hence, illegal IMO. In a similar fashion that the state has incorporated USPAP by reference into the state Code making a violation of USPAP illegal. That way the state AG can get involved in appraisal violations and enforcement.