YES most of your team members (specially the most effective / capable ones !) prefer to Work From Home (WFH). 
Perhaps enjoying the opportunity of a physical team meeting from time to time, but generally loving the autonomy to decide when & where to work from. 
Dreading anyone looking over their neck checking whether they are sitting there, only apparently working hard until late.

Also YES many of your team members might instead return to the office but for a very bad reason:
for Fear Of Missing Out (FOMO).

If your team is driven by fear of only being recognised because of their physical presence, you definitely have an ineffective motivation structure that is not able to recognise and reward star players. 

If you need people to come to the office for fear of missing out on rewards, you could be a much better leader.
You perhaps could do more to convey the reason why they should give their best.
You should accompany people individually to make them feel autonomous and able to do their best work without someone controlling / rewarding figure looking physically around them.

Additionally a team driven by fear, will rarely innovate. 
Imagine that lack of innovation today, in this era!
A sure recipe for failure.

Is fear the only way you are capable of using to “lead”?