Wipe down with acetone, then light grinding/heavy sanding of scale to prep the weld area is quicker/cheaper/faster than having to deal with porosity, etc.--after welding. Mill scale comes in a wide variety of types/thickness and contaminants--as you've surmised.
Often one can 'get away' with no...