On my CA server (Ubuntu 11.04) I have libogg0 package installed, and ffmpeg is compiled with libvorbis enabled:
ffmpeg
FFmpeg version 0.6.2-4:0.6.2-1ubuntu1, Copyright (c) 2000-2010 the Libav developers
built on Mar 22 2011 15:35:22 with gcc 4.5.2
configuration: --extra-version=4:0.6.2-1ubuntu1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --enable-shared --disable-static
...
I can't find any package for "ogg" support in ffmpeg, so I suppose "vorbis" is sufficient.
Though, when trying to upload an ogg audio clip to CA, I get the warning "Unknown file type not accepted by media"...