There are many times when we have to make a choice between how much to display on a form versus it is taking too long to load.
This is a complex question to answer as there is no rule of thumb to guide and answer this question. However there are few key considerations that must be made while deciding on form layout.
It is always best to start the form design as agile as possible. Take the client in the journey with you and help them make the decision in terms of what fields to display on the form. If the field is not one of the fields that the client wants when they open the form, do not display on the form. There are other ways to get the data like Advanced Find.
Do not overload the form with a lot of scripts. More complex and convoluted the script, more time it will take to load the form. There are new features available to avoid a lot of scripts / rules on the form. Consider using different forms for different roles.
Avoid loading a lot of related entities. This is a lot of times a main killer for the performance. It is great from the user experience perspective to get all the data on the form but loading more than what the client needs is overkill. Users can always get to the related entities from navigation pane.
Do not have long running synchronous processes. It is never a good idea to write a synchronous process for things like external fetch. It is always good idea to write a background / asynchronous plug-in for this to make sure client does not wait for the form to load / save for a long time.
Hope this helps.