Nope. There are several ways for a property to be out of compliance. (Besides - 'illegally zoned' would only apply if the city council passed the zoning law illegally). Out of compliance can mean a non-conforming lot, a non-conforming use (SFR being used as a commercial building), a non-conforming structure (built without permits, or remodel without permits, like splitting a house into 2 units and adding a 2nd kitchen).
BTW, California has a law that requires municipalities to allow an accessory unit in SFR land, providing it meets certain size requirements and the owner lives in one of the units. So, in your case, you have to find out if it meets the state requirements, and if it was permitted. If so, it's probably legal. If not, it's a teardown. Appraise at HBU, and cost to cure for the teardown (or if it's attached and the structure was permitted, just not the current configuration, they might be able to convert back to SFR.)