Also a few more things to check:
- Make sure the motors are connected in the correct order and spinning in the correct directions
- Make sure the propellers are on the correct way around
- Make sure the flight controller orientation is set correct in the software
- Ensure you have calibrated your Gyro and Accelerometer correctly (as per @Luca suggestion)
- Check radio calibration and that there is no mixing setup on your radio
- Make sure your ESC have all been calibrated to have the same PWM range.