Posts Tagged ‘Amazon’

I figured I would pass along some info I have collected while playing around over on the free services of Amazon AWS. One of the nifty things about AWS is that they offer a number of ‘free tier‘ services for trying out what they offer as well as getting familiar with the environment.  (Other services can be used with minimal expense for low-level usage – just pennies per month)

On such free tier service is the EC2 virtual server environment, which supports a number of operating systems including Amazon’s linux, Ubuntu, Windows, and other operating systems on a 1Gb memory, 8Gb storage* virtual server. (free services can also add elastic storage to increase the 8Gb if needed)

One problem you’ll run into right away is that 1Gb of memory is a small amount when using some modern tools. No problem, you can create virtual memory with a swap file using some of that free tier storage space. However, the instructions on the AWS documentation pages assumes a system larger than the free tier and won’t work as written.
The first problem will be encountered right off the bat. As with most other static-swapfile implementations, they create a static file using dd in conjunction with /dev/zero to create the initial file. The problem is, their example uses a blocksize equal to (1Gb) the total memory of the EC2 free tier instance. So instead use a smaller blocksize and a bigger count.  I used 128M which uses a count of 8 for 1Gb in size. So for example, to create a 1 1/2 Gb swap file:

$ sudo dd if=/dev/zero of=/swapfile bs=128M count=12

The rest can be done as documented: How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?

Another useful diddy is the ability to mount an S3 bucket as a local fuse device. This is made possible with >s3fs. Follow their instructions then you can even mount the device using an fstab entry. Example:

s3fs#mybucket /media/mybucket fuse _netdev,use_cache=/tmp,allow_other,default_acl=public-read,endpoint=us-east-2,dbglevel=info,uid=33,gid=33,mp_umask=002,multireq_max=5,url=https://s3-us-east-2.amazonaws.com 0 0

Replace ‘mybucket’ with the name of your bucket after the s3fs# and pick your own mount point as the second option. A few notes on some of the other options. You can exclude the default_acl if you want to use the bucket default. In this case I use ‘public-read’ to mark any newly created files as publicly readable. The combination of the endpoint and the url options makes sure you are accessing a region-specific bucket without error. uid and gid can be set to whatever user you wish. (in this case I am setting both to www-data). mp_umask is a reverse mask of the file permissions when mounted. You can also vary the debugging level as needed when troubleshooting. (be sure to read the instructions on setting up the /etc/passwd-s3fs file)

One thing to note is that you will need to assign permissions (likely an IAM role) to your EC2 instance that has access to S3 or at least your S3 bucket depending on what specific functions you want to do over the virtual-fuse connection. (e.g. give S3 createObject/delObject permissions if you want write access, getObject to read, etc – one easy way to do this at first is to add the S3FullAccess policy until you learn specifically what you need)

If you plan on playing with lambda, especially with custom runtime environments, the EC2 free tier is also useful as you can create an Amazon linux instance to compile the runtime environments to work on lambda. (NOTE: lambda uses the original version of Amazon linux, not Amazon linux 2. You’ll get GLIBC errors if you don’t downgrade the gcc environment on other platforms to match the libraries used on the original amazon linux)

It should be noted that the free tier servers on EC2 give you 32 days per month total. So this is more than enough to run a single free-tier-eligible instance non-stop all month, or to run multiple instances intermittently in so long as the combined usage does not exceed 32 hours. For example, I run an Ubuntu non-stop, but have an amazon linux instance for creating the runtime configurations that I only power up when needed. That gives me 24-48 hours of usage per month on the second instance without exceeding the total usage. (bonus time in February!)

Read Full Post »