There's not a lot of documentation about this on the webernet, so here goes. (Note, I'm on a 64 bit processor, so make sure you download the right packages for your machine).

First, I had to remove my GHC:

sudo apt-get remove ghc

Unfortunately the newest version of GHC in the Ubuntu apt repositories is 6.8. Haskell Platform needs 6.10 (get the newest, and appropriate for you from GHC).

wget http://www.haskell.org/ghc/dist/6.10.3/ghc-6.10.3-i386-unknown-linux-n.tar.bz2
bunzip2 ghc-6.10.3-x86_64-unknown-linux-n.tar.bz2
tar xvf ghc-6.10.3-x86_64-unknown-linux-n.tar
cd ghc-6.10.3

Now configure and install:

./configure
sudo make install

(You might have to install libgmp3-dev).
Next download the Haskell Platform:

wget http://hackage.haskell.org/platform/2009.2.0.1/haskell-platform-2009.2.0.1.tar.gz

Untar:

tar xvzf haskell-platform-2009.2.0.1.tar.gz
cd haskell-platform-2009.2.0.1/

And configure:

./configure
# I GET THIS ERROR: configure: error: The OpenGL C library is required
# fixed all by installing the following deps
sudo apt-get install libgl1-mesa-dev
sudo apt-get install libglc-dev
sudo apt-get install freeglut3-dev
sudo apt-get install libedit-dev
sudo apt-get install libglw1-mesa libglw1-mesa-dev
./configure
#now build
sudo make install

#finally
cabal update

Note, as of today I originally tried GHC 6.10.4. Under this, cabal failed, even on list with "cabal: 1: file not found"