AWS IAM Policy Statement to Register and Deregister Instances with/from ELB

E.g.

{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "Stmtxxxxxxxxx",            "Effect": "Allow",            "Action": [                "elasticloadbalancing:RegisterInstancesWithLoadBalancer",                "elasticloadbalancing:DeregisterInstancesFromLoadBalancer"            ],            "Resource": [                "arn:aws:elasticloadbalancing:aws-region:aws-account-id:loadbalancer/*"            ]        }    ] }

AWS IAM Permissions to Create Tags

Had a hard time getting this to work, and it turns out that only permissions to the "ec2:CreateTags" action is needed, however, the moment the resource is constrained (even to all resources within a specific region), I was unable to tag an EC2 instance. I was only able to tag an EC2 instance when the "Resource" was set to *

{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "Stmtxxxxxxxx",            "Effect": "Allow",            "Action": [                "ec2:CreateTags"            ],            "Resource": [                "*"            ]        }    ] }

Ansible Best Practices: Directory Layout

Here: http://docs.ansible.com/ansible/playbooks_best_practices.html#directory-layout

AWS CloudWatch Logs: Create Log Stream via Command Line

aws logs create-log-stream --log-group-name groupName --log-stream-name streamName

Ansible: Running a Command on the Local Machine

To run an command on the shell of Ansible machine (i.e. instead of the target machine), use the local_action module:

local_action: command xxxxxxxxx

Ansible: EC2 Instance ID Variable

{{ ansible_ec2_instance_id }}

MongoDB: Query for Documents Based On Array Size

db.collectionName.count({"arrayName":{$exists:true},$where:'this.arrayName.length>3'})

Reference: http://stackoverflow.com/questions/7811163/query-for-documents-where-array-size-is-greater-than-1

Ant Java Compilation Classpath

If there are many jar files in the classpath, instead of specifying all of them, include all jar files in a folder this way: http://stackoverflow.com/questions/6103516/ant-adding-multiple-jars-in-classpath-dynamically

Amazon Linux Install Ansible

The YUM method won't work, due to differing python versions.

Use pip instead.

sudo pip install ansible

Passing Environment Variables into Java from Command Line

java -DvarName1=varValue1 -DvarName2=varValue2

Pentaho Data Integration PostgreSQL Bulk Loader Gotcha

The password you key in the connection won't work. It only works when authentication is disabled for the server Pentaho is running on.

Reference: http://wiki.pentaho.com/display/EAI/PostgreSQL+Bulk+Loader#PostgreSQLBulkLoader-Setupauthentication

Cheat Sheet on 16 Analytics Techniques

Here it is: http://www.forbes.com/sites/bernardmarr/2016/02/04/the-18-best-analytics-tools-every-business-manager-should-know/#ebb1f7d2c4ad

MongoDB "update" Only Updates One Entry by Default

The "update" command only updates the first matching entry by default. To update all entries, there is a third parameter i.e. "options" where you need to set "multi: true" e.g.

db.coll.update({},{$set:{"attr":”value”]}},{multi:true})

Reference: https://docs.mongodb.org/v2.4/reference/method/db.collection.update/

AWS IAM: EC2 Full Access to Region

Example:

{    "Version": "2012-10-17",    "Statement": [        {            "Sid": "xxxxxxxxxxx",            "Effect": "Allow",            "Action": [                "ec2:*"            ],            "Resource": "*",            "Condition": {            "StringEquals": {                "ec2:Region": "us-west-2"            }        }      }    ] }

Reference: http://stackoverflow.com/questions/18112784/iam-allowing-a-user-to-access-everything-for-ec2-on-a-region

AWS S3: Setup IP Based Access Control

Use the "Condition" attribute:

"Condition" : {    "IpAddress" : {        "aws:SourceIp": "xxx.xxx.xxx..xxx/32"    }, }

More examples can be found in following references:

• http://docs.aws.amazon.com/AmazonS3/latest/dev/amazon-s3-policy-keys.html
• https://pete.wtf/2012/05/01/how-to-setup-aws-s3-access-from-specific-ips/

Techcrunch 2015 Year End Analysis of the Analytics Market

Great read!

http://techcrunch.com/2016/01/09/the-year-in-analytics/

Disable Android File Transfer on MacOS

I seldom transfer file so this is an unnecessary annoyance.

Steps here: https://cooltrainer.org/taming-android-file-transfer-on-mac-os-x/

MongoDB Security

By default, security and authentication is disabled. Create the administrator account before switching on authentication. Enter the console of the admin DB:

mongo admin

Add the administrator user:

db.createUser({user: "adminUserName",pwd:"adminUserPassword",    roles: [ { role: "userAdminAnyDatabase", db: "admin" }, { role: "readWriteAnyDatabase", db: "admin" }, { role: "dbAdminAnyDatabase", db: "admin" }, ] })

Enable authentication by editing the "/etc/mongod.conf" file
Uncomment this line:

auth=true

Restart MongoDB. Enter the console of the admin DB and you'll find that you'll need to authenticate before performing any operations:

db.auth({user: "adminUserName",pwd:"adminUserPassword"})

Now you should be able to add a user for the app database containing your data, and make this user the owner of the app database:

db.createUser({user: "dbUserName",pwd:"dbUserPassword",    roles: [ { role: "dbOwner", db: "dbName" }] })

Note: this needs to be done inside the app database

To enable remote login, comment out the "bind_ip" in "/etc/mongod.conf".

Note: Steps tested on MongoDB 2.6.x

AWS Ant Upload to S3

Use AWS Ant Tasks, an open source library released by AWS in the AWS Labs.

To use it, define the taskdef:

<taskdef resource="taskdefs.xml" classpath="~/.ant/lib/aws-java-sdk-ant-tasks-1.2.3.jar" />

Example of upload task:

<target name="deploy-s3" depends="war">    <upload-to-s3 bucketName="bucket-name" keyPrefix="folder-name/" continueOnFail="false" awsAccessKeyId="${aws.access.key}" awsSecretKey="${aws.secret.key}">        <fileset>            <!-- define files here -->        </fileset>    </upload-to-s3> </target>

Note: As at current time of writing it's broken with Java 8. You'll have to run Ant with Java 7.

References:

• https://github.com/awslabs/aws-ant-tasks
• https://java.awsblog.com/post/Tx1VW8X2W3775XC/AWS-Ant-Tasks

AWS Command Line Download S3 File

If you want to use the save credentials in ~/.aws/credentials, use the "aws s3 cp" command:

aws s3 cp s3://<bucket-name>/<folder-name>/<file-name> <local-file-name>

Reference: http://docs.aws.amazon.com/cli/latest/reference/s3/cp.html