Do I need to be connected to the internet or a cellular network to make a payment with Samsung Pay?

No, an active internet connection is not required to make in-store purchases, but your device should connect to the internet at least once per day to ensure that Samsung Pay stays up to date and remains active.  Samsung Pay does require an active internet connection to activate the app or add a new card.