Life in the clouds, Googie style (1962).

Using AWS

by: burt rosenberg
at: university of miami
date: january 2021

Overview

From the 202 edition of this course (Jan-May 2020) the course has gone fully to programming on AWS EC2 instances. You access the EC2 instance from any computer using ssh, scp and sftp, using public-key style authentication.

Many platforms are possible for accessing your instances. Linux and MacOS tend not to be problems.

• Linux is actually the kernel of a Unix operating system that is more GNU system code. Then distros, which are packaging mechanisms, present various forms of this combination under the shortened name of Linux. These tools are native.
• In 2001, Apple introduced its next generation operating system OSX that combined three things: FreeBSD unix, the MACH kernel, and a NextStep windowing (Aqua) and an object-oriented IO system (IOKit). These tools are near native, and Brew will bridge any gaps.
• Windows might pose the biggest problem, as it started out very proprietary and was not as advanced in the free (free as in beer) style of marketing as was Linux. As Apple transitioned to OSX, Windows transitioned to Windows NT, a VMS derived kernel with compatibility layers, particularly for Microsoft's own Win32 API. At that time other compatibility layers were planned, but when Microsoft pulled out of its collaboration with IBM, those were put on the back burner. However the WSL (Windows Services for Linux) now seems to now be implemented to a point were a standard, hassel-free ssh is available.

Amazon Web Services

You will create a computing instance from,

• a compute instance, (EC2),
• and storage instance, (EBS),

If you do not have one, you will open an account with aws.amazon.com. Back when I did it, I just used my "retail" amazon credentials, but now things are more complicated. The aws account your first create is the root account, and you then create users in the IAM service, and assign policies to those uses, for instance the AdminstratorAccess policy, which gives control over everything except billing. It is also possible to add DUO MFA authentication.

It is important with MFA to have a few MFA's. To recover from MFA lose or confusion is hard. Passwords also have to be accurately memorized, as amazon might be very coy as to what element of the login it did not like. Passwords can be reset if you email information is correct, which makes less dangerous that they are a blocking sort of authenticator.

If you go the route of IAM and MFA, I set up these two, so if one does not work the other might,

• DUO: an app or token that does TOTP one time password symmetric MAC type stuff.
• Yubikey: a FIDO device the asymmetric encryption signatures type stuff.

Creating an EC2 Instance

There are three issues to talk about,

• The AMI (Amazon Machine Image). Ubuntu 22.04 LTS, so that we are all on the same platform.
• Free tier eligible. For the first year after enrolling in AWS there is an allowance called free tier. We want to remain in that even if out of the first year as it is among the cheapest option.
• The Key Pair. Rather than passwords, you use ssh public key to sign in, using a key pair. You will be given the option to create a key pair, or use one already created. When you create a key pair it only retains the public half, and downloads the private half. Find and organize the private half; it is like a password.

    ssh -i "aws-csc424.pem" ubuntu@100.24.236.162

1. The username, ubuntu
2. The IP address 100.24.236.162, ymmv
3. The name of the private key file aws-csc424.pem, ymmv

raritan:keys ojo$ pwd
/Users/ojo/svn/classes/csc424-202.svn/class/keys
raritan:keys ojo$ chmod go-r aws-csc424.pem 
raritan:keys ojo$ ls -lt
total 16
-rw-------@ 1 ojo  staff  1692 Apr 19 15:50 aws-csc424.pem
-r--------@ 1 ojo  staff  1692 Apr 19 11:56 csc424-keypair.pem
raritan:keys ojo$ ssh -i "aws-csc424.pem" ubuntu@100.24.236.162
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-1057-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Sun Apr 19 20:26:41 UTC 2020

  System load:  0.08              Processes:           86
  Usage of /:   13.6% of 7.69GB   Users logged in:     0
  Memory usage: 14%               IP address for eth0: 172.30.1.238
  Swap usage:   0%

0 packages can be updated.
0 updates are security updates.



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@ip-172-30-1-238:~$ sudo apt-get update
Hit:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]

...

Get:27 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [7176 B]
Get:28 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2764 B]
Fetched 18.6 MB in 4s (4861 kB/s)

ubuntu@ip-172-30-1-238:~$ sudo apt-get install subversion build-essential
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu cpp cpp-7 dpkg-dev fakeroot g++ g++-7 gcc gcc-7 gcc-7-base gcc-8-base
  libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libapr1 libaprutil1 libasan4 libatomic1 libbinutils
  libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-7-dev libgcc1 libgomp1
  libisl19 libitm1 liblsan0 libmpc3 libmpx2 libquadmath0 libserf-1-1 libstdc++-7-dev libstdc++6 libsvn1 libtsan0 libubsan0
  linux-libc-dev make manpages-dev
Suggested packages:
  binutils-doc cpp-doc gcc-7-locales debian-keyring g++-multilib g++-7-multilib gcc-7-doc libstdc++6-7-dbg gcc-multilib autoconf
  automake libtool flex bison gdb gcc-doc gcc-7-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan4-dbg
  liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx2-dbg libquadmath0-dbg glibc-doc bzr libstdc++-7-doc make-doc
  db5.3-util libapache2-mod-svn subversion-tools
The following NEW packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu build-essential cpp cpp-7 dpkg-dev fakeroot g++ g++-7 gcc gcc-7 gcc-7-base
  libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libapr1 libaprutil1 libasan4 libatomic1 libbinutils
  libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-7-dev libgomp1 libisl19
  libitm1 liblsan0 libmpc3 libmpx2 libquadmath0 libserf-1-1 libstdc++-7-dev libsvn1 libtsan0 libubsan0 linux-libc-dev make
  manpages-dev subversion
The following packages will be upgraded:
  gcc-8-base libgcc1 libstdc++6
3 upgraded, 45 newly installed, 0 to remove and 73 not upgraded.
Need to get 45.7 MB of archives.
After this operation, 175 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y

Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic-updates/main amd64 gcc-8-base amd64 8.4.0-1ubuntu1~18.04 [18.7 kB]
Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libstdc++6 amd64 8.4.0-1ubuntu1~18.04 [400 kB]
Get:3 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libgcc1 amd64 1:8.4.0-1ubuntu1~18.04 [40.6 kB]

...

Setting up build-essential (12.4ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
ubuntu@ip-172-30-1-238:~$ 

Stopping and terminating the EC2 instance

• Stopping an instance: This deletes the compute instance but keeps the EBS storage. You and restart the instance, which really is a new instance, but it rejoins the EBS storage so you pick up where you left off.
• Terminating an instance: This deletes the computer instance and also deletes the EBS storage. You are done, there is nothing more that will be billed. It's all gone.

When stopping and restarting, the IP address might change. Elastic IP allows you to make sure stopping and restarting gives you the same IP address, however, unused elastic IP's are charged at a punishment rate. So for this class, ignore elastic IP's.

Let's talk $$$'s

The Free Tier is for new users their first 12 months. Free is,

• 750 hours per month of Linux, RHEL, or SLES t2.micro or t3.micro instance dependent on region
• 30GB of Storage, 2 million I/Os, and 1GB of snapshot storage with Amazon Elastic Block Store (EBS).

• A running t2.micro is $0.0116 per Hour ($33 per semester per machine).
• If you keep an EBS volume off line, it's $0.10 per GB-month ($3 per semester per 8G machine).
• EBS Snapshots are $0.05 per GB-month of data stored.
• An Elastic IP address not associated with a running instance is $0.005 per hour ($14 per semester per machine).

Remote editing

The ssh command uses the .ssh/config to configure shortcuts. Place in your conf file write something like,

		Host aws-csc424-s
		Hostname 18.208.120.70
		User ubuntu
		IdentityFile ~/csc424.svn/burt/keys/aws-csc424.pem

Then you can log into you machine with:

    ssh aws-csc424-s

   scp hello.c aws-csc424-s:

A good programmers editor for Mac that includes SFTP is BBEdit. Using this, there is a save-to, open-from sftp option. The BBEdit software uses the .ssh/config options, so the name of the server will be (in this case) aws-csc424-s.

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

author: burton rosenberg
created: 18 apr 2020
update: 18 jan 2024