Yeah, as already mentioned there is a fine balance between bigger battery and longer flight times. But in general it is just get your quad as light as possible, and use the biggest propeller you can fit on your motors. And then select a battery that has just enough C rating that you will need. But with FPV quadcopters the general aim is to get something as fast as possible and only caring about efficiency second. To build a long duration quadcopter it will generally not be as nimble as a racing quad, and to get seriously long flight times you will probably need to go with something that uses at least 15 inch propellers.
I wrote an article about how to go about choosing a motor/propeller / battery combo if you wanted to read abit about it - How to choose the right motor for your multicopter drone