Short answer: Usually yes. Only a plan with fewer than 50 participants that the employer administers itself falls outside the HIPAA definition of a group health plan; if a carrier, TPA or other outside party administers it, it is covered at any size.
The HIPAA definition of a group health plan covers an ERISA welfare plan providing medical care, insured or self-insured, that either has 50 or more participants or is administered by someone other than the employer that set it up.
So the small-plan exception is narrow. A fully insured plan is administered by the carrier, and a level-funded plan, HRA or health FSA is typically run by a TPA, so each of those is a covered entity however small the employer. Only a plan with fewer than 50 participants that the employer truly runs in-house (for example, a tiny MERP the business owner reimburses directly) is outside the rule.
Being outside HIPAA does not make the information unprotected: state law and other federal laws can still apply.