Installing a development server

I got myself in a mess by using Ubuntu's aptitude package to remove a package I accidentally install (I installed full emacs instead of the no X version, so it install X which I don't want).  Unfortunately when it removed packages that no longer had dependencies because emacs was no longer there, it removed a LOT of needed packages.  Including reboot, which the system said it needed after all the packages were removed.  I figured that I had only spent a few hours in setting up the server (codemonkey) that I would just do a reinstall of everything.  So I'm going to document step-by-step the process here.
Of course first thing is I get my trusty Ubuntu 14.04 CD, stick it in the CD drive and hit the reboot button.  It comes up and asks me several times if I speak english (and what kind of english, I suppose if I said I spoke British english it would use British spellings like colour instead of color). It asks if it wants it to detect the type of keyboard I have, which I chose the default no, as that's gotten me in trouble before (it asks if I have an umlaut, and I though it was asking if I have a double quote...duh). Next it ask which of the three network adaptors I have is the primary, and I pick the first one (I have to set up the other two manually later).  It uses the DHCP system to auto setup the network, so it can download packages from the Internet (you didn't think the whole system would fit on a CD did you?) next it asks for the system name, and some figured out that it was Codemonkey, so I accept that.  Next it asks for the first user name, as it needs at least one user, that is marked as sudo capable.  I entered my name, user name, and usual password.  It asks me if I really want to use my weak-ass password, to which I respond yes, as I prefer not to be judged by my computer.  Then it asks if I live in Los Angeles so it can set my timezone, and I lie to it and answer yet (although I HAVE lived in Los Angeles, I have fortunately since escaped).
The next step is to partition my hard drive.  Since I already have a system, albeit broken, it gets very concerned and now I have to take non-default answers, and it ask me again and again, do you really want to write over this fine system? Are you really really sure? Ok, this is your last chance! Is that your final answer? YES, YES, YES, YES.  So finally I musters enough confidence to format my hard drive with the default set-up...swap...boot...LVM.  Then it asks if I use an HTTP proxy to access the Internet (as if anyone really does that anymore).  It asks how I want to handle upgrades, and my typical response is to automatically install security patches.  It's not really automatic, as sometimes it requires a reboot of my system, so I have to logon occasionally and see the 'reboot required' message pops up and reboot as needed (It's not as bad as Windows which needs a reboot if you look at it wrong).
Next it presents me with some typical server packages that I might want to install.  I pick OpenSSH, LAMP, Postgres, and Mail.  I can't imagine not using ssh, as it is my main way of communicating with the server from my Mac in my easy chair, I need LAMP for the Apache server, Postgres because Jira and Confluence need it, as Oracle makes it difficult to use MySQL which I imagine they're trying to kill so they can get back to the business of selling software instead of giving it away.  And I need mail so that Hudson can send me mail everytime I break something.  It asks some configuration questions for those packages, so I enter the MySQL root password, the domain name for my mail server and basic setup for my mail server, and the common name for the self-signed certificate that I will be using, which I just enter as 'codemonkey'.  If I create a real SSL certificate (which now costs 50 dollars a year from godaddy), I'll have to pick one of the external host names that all point back to codemonkey.  It's a real pain that you have to have a separate official certificate for every hostname, I wish I could just have a single certificate for the domain that would cover all the hosts.  You may have seen warning from sites, if you just enter the domain name instead of 'www' dot domain name and it says the the certificate was issued to www.blahblahblah because they only got a certificate for the www host and not the domain.  Fortunately only the browser cares if the name on the certificate is right, all other programs blithely accept the certificate even if the common name is hackers.inc, and just use it to create the secure connection.  Since I probably won't shell out 50 more bucks a year to godaddy, anyone that tries to access my website will have to get past the browsers security warning about trusting someone called 'codemonkey' and add my certificate to their list of trusted certificates.  Now I'll get off my soap box and back to the install.
It want's to know if I want to add the boot program 'Grub' to my master boot record, which I answer the default yes.  I like to know that my master boot record on my hard drive points to grub so I can make sure I'm not compromised by a rootkit virus which are very hard to detect and eliminate.  Finally it ejects my CD so I can remove it and not accidentally re-install my system again, and continue to reboot into my new system. When the system reboots, I logon, and find that it needs a couple dozen updates, about 10 of which are security updates.  So I run the command:

sudo apt-get upgrade

 And with that my system is setup as a basic server.  Now to configure ssh so that all my (Unix/Linux) computers can talk to each other.  I do have one Unix system, my Mac.  Surprising to many people that Macs are the antithesis of command line computer systems, one can drop down into a command shell and viola! Unix.  Ha ha, fooled ya with my fancy UI.  Actually it's part of what makes Macs so resilient and secure.  From now on, all my new computers will be Macs, Windows can go away and lick its vulnerabilities.
To make your systems talk to each other, they have to exchange keys.  It's like giving a spare house key to a trusted relative, so they can turn off the water you left on when you go off on vacation.  At this point I can do the rest of the install from my Mac.  But first, I should start some downloads (which I overconfidently deleted from my system after install them).  I need four pieces of software to make my system complete: Jira, Confluence, Jenkins, and Nexus.  Jira and Confluence are 10 dollars each (when I finally get my real licenses, I still have a week on my eval license).  Jenkins and Nexus are free, although they have 'pro' versions incase you're a moron or middle management (middle management being a subset of the moron class (excepting my own manager in case he's reading this)), and need help installing or configuring the systems.  Here's the URL's, you'll have to navigate to the download pages yourself, as they often change. (Jira me to down load the OSX version since I'm on my Mac, and I had to reassure it that I really needed the linux 64-bit installer)

https://www.atlassian.com/software/jira
https://www.atlassian.com/en/software/confluence
http://pkg.jenkins-ci.org/debian/
http://www.sonatype.org/nexus

Jenkins isn't actually a download, it's instructions on how to hack your system to be able to automatically download from their repository.
So now I need to do the key exchange.  Technically I could have simply saved the .ssh directory on my server that I reinstalled, and replaced it after the reinstall.  I've never actually done that, so I can't guarantee that it would work though.  I open a command prompt on my Mac (if you don't know how to do this, perhaps you shouldn't be running a server).  I have already set up my router to have DHCP reservation for my servers by MAC address, so the names available instead of using IP addresses.  If you don't have such a system in place you may have to use IP address.  Good luck with that. so here's the conversation:

megamac:~ randalkamradt$ ssh codemonkey

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!

Someone could be eavesdropping on you right now (man-in-the-middle attack)!

It is also possible that a host key has just been changed.

The fingerprint for the RSA key sent by the remote host is

xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.

Please contact your system administrator.

Add correct host key in /Users/randalkamradt/.ssh/known_hosts to get rid of this message.

Offending RSA key in /Users/randalkamradt/.ssh/known_hosts:7

RSA host key for codemonkey has changed and you have requested strict checking.

Host key verification failed.

I guess my Mac got suspicious about codemonkey not having an RSA key yet (as I'm about to set it up). If this is your first install, you won't get it, I only get it because I reinstalled the system.  If you don't get this, you can skip to the next paragraph.  Here's where we have to do some hacking.  'cd' to the '.ssh' directory (don't worry if you don't see it, it's hidden).  Edit the authorized_keys file, find the line that starts with 'ssh-rsa' and ends with your servername, and delete it.  That was the old key and we need to generate a new one.  Next, edit the known_hosts file.  You may find a lot of hosts here if you've got a githup account for instance, or other online accounts that require some kind of ssh access.  You're going to find the line that starts with your servername (or server IP address or both separated by a comma)  If the keys look like gibberish, that's good, they are basically very long passwords that make them very hard to guess.  If you're using emacs like me and find the '~' backup files annoying, you should delete them now to avoid further annoyance.  You will have to do this with any other servers that have exchanged keys with the old server as well.  In case your key in known_hosts doesn't have the server name in it, you can look at the message above and find the line 'Offending RSA key in /Users/randalkamradt/.ssh/known_hosts:7' and use the number at the end in place of the servername.

Now when you ssh to the host you should get a new message about it not being recognised:

megamac:.ssh randalkamradt$ ssh codemonkey

The authenticity of host 'codemonkey (192.168.1.132)' can't be established.

RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx;xx:.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'codemonkey,192.168.1.132' (RSA) to the list of known hosts.

But now codemonkey is recognized by megamac (the name for my macmini) and you should be able to logon.  Because I use the same username on all systems, I don't have to specify a username, otherwise I'd have to use randalkamradt@codemonkey.  If I want to exchange keys with another user on codemonkey, I'd have to prepend the username@ to the servername. Now I'm in codemonkey, I have to create a public and private key. This is for PSK authentication, where we create a public and private key, issue the public key to anyone and everyone, and they can encrypt anything they send with the public key, and in can only be decrypted with our private key (which we should keep safe and never give away)  Here's how to do that:

randalkamradt@Codemonkey:~$ ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/randalkamradt/.ssh/id_rsa): 

Created directory '/home/randalkamradt/.ssh'.

Enter passphrase (empty for no passphrase): 

Enter same passphrase again: 

Your identification has been saved in /home/randalkamradt/.ssh/id_rsa.

Your public key has been saved in /home/randalkamradt/.ssh/id_rsa.pub.

I don't use a passphrase, and that leave me vulnerable in case someone breaks into my house and steals all my private keys.  But they'd have to get past my attack cat, Katie, first.  I suppose I could also accidentally send people my private key, but I don't foresee that happening.  Not having a passphrase means that programs on the two computers can communicate without me entering the passphrase or storing my password somewhere.

Next comes the key-exchange.  You have to do this on both sides.  Here's codemonkey exchanging with megamac:

randalkamradt@Codemonkey:~$ ssh-copy-id megamac

The authenticity of host 'megamac (192.168.1.102)' can't be established.

RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.

Are you sure you want to continue connecting (yes/no)? yes

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

Password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'megamac'"

and check to make sure that only the key(s) you wanted were added.

That should be the last time I have to enter megamac's password to access him from codemonkey.  Notice that I had to add megamac to the list of known hosts.  Now I have to do the reverse, giving codemonkey megamac's public key.  The Mac version of Unix doesn't have the ssh-copy-id command, or I haven't found the right command, but I can do it the old fashion way: copy my public key from ~/.ssh/id_rsa.pub, ssh to codemonkey, and append it to ~/.ssh/authorized_keys.  Now all my computers are speaking to each other, it's time to install the software.

All my downloads are complete, but now I have to transfer them to codemonkey.  For this I can use scp:

megamac:~ randalkamradt$ scp atlassian-confluence-5.5-x64.bin codemonkey:~

atlassian-confluence-5.5-x64.bin                           100%  237MB   7.9MB/s   00:30    

megamac:~ randalkamradt$ scp atlassian-jira-6.2.3-x64.bin codemonkey:~

atlassian-jira-6.2.3-x64.bin                               100%  204MB   8.9MB/s   00:23    

megamac:~ randalkamradt$ scp nexus-2.8.0-05-bundle.tar.gz codemonkey:~

nexus-2.8.0-05-bundle.tar.gz                               100%   43MB   8.7MB/s   00:05    

And that should transfer them to codemonkey in my user directory.  I'll finish up the work of installing the software at another time, as I need a break for now.  But what we've done so far is substantial, we've installed a server with two databases, an Apache http server (go ahead and try http://servername and you'll see the 'it works' page) a mail server, and a SSH server.  If you want these to be accessible to the Internet you'll need to punch holes in your router firewall (assuming you have a router) at ports 22, 25, 80, and 443 (if you want https).  Most of these services are pretty secure, so opening up these ports is pretty safe, although I might replace the Apache 'it works' page with a more minimal 'it works' page, and don't punch holes for the databases, as they are not yet secure, and you probably don't want to expose them anyway, they're for internal use only.

To be continued...