I assume that you ran through the firstboot scripts and set your root password etc. I use Bodhi Linux on my netbook (which is based on Ubuntu 10.04) and I haven't ever had issues with SSH connections working OOTB. Hard to say what might be going on...

Can you log in ok locally? Assuming you can, have a look what the SSH service is doing:

service ssh status

Assuming that is running (if not try 'start' instead of 'status') then check it is listening:

netstat -l |grep ssh

it should return something like this:

tcp        0      0 *:ssh                   *:*                     LISTEN     
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN     

At least that's what the TKL server (Core) that I just checked returns... If things all seem ok from that angle, then maybe try rebooting and to be sure, double check the appliance IP (ifconfig).

TBH though unless you can pin the issue down to something external (networking/firewall/etc) I'd be a little concerned that it doesn't work OOTB (as it should) and would encourage you to check the integrity of the image you downloaded (have a read here).

Out of interest by stopping the SSH service I could recreate the "connection refused" message.

BTW SFTP is available by default OOTB (although I'm guessing it won't work in your instance seeing as SSH isn't working...)