My reading is that in addition to protecting you when unarmored, no matter who you're fighting, the Soft Armor from Tempered is active when you're fighting lesser foes IN ADDITION TO any armor you're wearing.