Do I need to connect my hardware wallet to receive ADA?

No, as no confirmation is needed from your Ledger to receive ADA. You only need to provide your receiving address to the sending party.