Which version of InDesign and OS?
How are you creating your PDFs ?
Presumably the PDF is for emailing etc, not for printing.
You need to experiment with different settings, comparing size against quality. Look at the size of your images, can they be reduced? Try reducing the PPI say to 150 in the PDF settings dialogue. Try out the Reduced Size PDF facility.