What to turn off, and how?

The following code works well for me, the ATmega328P draws < 0.5µA current while in power-down mode (running on 2 AA cells). Before calling goToSleep(), I ensure that all pins are either (a) defined as inputs with pullups enabled, or (b) set to a low level if they're output pins driving loads like LEDs or whatever. Be sure to provide a way to wake up, in this case I'm using pin change interrupts on any of several pins.

Of course this will not reduce current consumption of other circuit components like voltage regulators, etc., as CrossRoads correctly states.

#include <avr/sleep.h>
...
void goToSleep()
{
    byte adcsra, mcucr1, mcucr2;

    sleep_enable();
    set_sleep_mode(SLEEP_MODE_PWR_DOWN);
    PCICR = _BV(PCIE1);            //enable pin change interrupts 8-14
    PCMSK1 = _BV(PCINT8) | _BV(PCINT11) | _BV(PCINT12) | _BV(PCINT13);    //enable PCINT8,11,12,13 (PC0,3,4,5)
    adcsra = ADCSRA;               //save the ADC Control and Status Register A
    ADCSRA = 0;                    //disable ADC
    cli();
    mcucr1 = MCUCR | _BV(BODS) | _BV(BODSE);  //turn off the brown-out detector
    mcucr2 = mcucr1 & ~_BV(BODSE);
    MCUCR = mcucr1;                //timed sequence
    MCUCR = mcucr2;                //BODS stays active for 3 cycles, sleep instruction must be executed while it's active
    sei();                         //ensure interrupts enabled so we can wake up again
    sleep_cpu();                   //go to sleep
    sleep_disable();               //wake up here
    ADCSRA = adcsra;               //restore ADCSRA
}